ICDL 2007 Enabling Lifelong Human-Robot Interaction

Enabling Lifelong

Human-Robot Interaction

A Special Session of the Intl. Conf. on Development and Learning

July 12, 2007

Imperial College London

What are the applications of robotics critical to society?

It is unclear how the capabilities of current and future robots will meet needs of their human users over the course of time. As robots move beyond laboratories into real-world human environments, human-robot interaction will be increasingly longitudinal. The performance of personal robots, similar to personal computers, will be subject to the dynamic expectations of human users and evaluated over of the span of years. Such long-term interaction poses distinct challenges for the scalability of autonomous robotic systems.

Scalability highlights the need for enhanced robot learning and development. Specifically, how can robots scale to perform unknown tasks across different environments and hardware platforms according to the preferences of individual users? To meet this challenge, we must revisit basic issues in developmental robotics about innate mechanisms and adapting behavior. Can innate robot capabilities be crafted to sufficiently encompass the space of relevant tasks, environments, and platforms? Can these innate capabilities be formalized mathematically and interfaced with human decision making? Should learning and adaptation be the central means for scalability? Can appropriate learning methods be performed tractably over large datasets online? How will users produce training data without programming or a prohibitive burden? How could innate mechanisms be structured to permit abstraction and generalization by learning? Are there common concepts used in existing approaches to these issues? (What learning methods and representations are needed to allow for assimilation of knowledge over extended periods of time and from different perceptual mechanisms?)

Further, scalability for lifelong HRI raises questions about how to evaluate across the uncontrolled factors. Is evaluation solely in a laboratory setting still sufficient? What combination of quantitative, qualitative, usability, and longitudinal aspects are needed for evaluation? Given the overhead for experimental infrastructure, how can we realize platforms that enable truly normalized evaluation across different algorithmic approaches? For longitudinal studies, could such robots be feasibly deployed to lay users? Is robotics at a point where normalized evaluation is realistic?

This special session of ICDL will address the issues facing lifelong HRI including but not limited to the questions raised above. We will assemble a diverse group of researchers in and beyond robotics to explore the convergence of theories about human development, human-machine interfaces, machine learning, and robot engineering towards developing scalable autonomous robots.

Program Schedule (Tentative)

15:00-15:30 B. Kuipers (Invited Talk): Learning the Foundations for Life-Long Learning

15:30-15:45 M. Nicolescu, O. Jenkins, A. Stanhope: Fusing Robot Behaviors for Human-Level Tasks

15:45-16:10 -- Break --

16:10-16:40 R. Arkin (Invited Talk): Behavioral Development for a Humanoid Robot: Towards Life-long human-robot partnerships

16:40-17:00 A. Thomaz, C. Breazeal: Robot Learning via Socially Guided Exploration

17:00-17:20 M. Pardowitz, R. Dillmann: Towards Life-Long Learning in Household Robots: the Piagetian Approach

17:20-17:40 N. Koenig: Toward Real-time Human Detection and Tracking in Diverse Environments

17:40-18:00 C. Kemp (Invited talk): Manipulation in Human Environments

Organizer

Chad Jenkins (Brown University)
cjenkins@cs.brown.edu

Image credit: K. Adam White, kadamwhite at gmail.com

Confirmed Talks

Behavioral Development for a Humanoid Robot: Towards Life-long human-robot partnerships
Ronald C. Arkin (Georgia Tech)

Recently a significant research effort was conducted at Sony's Intelligence Dynamics Laboratory (SIDL), involving personnel from Georgia Tech, MIT, CMU, Osaka University, and SIDL, working towards the implementation of a theory of designed development for a humanoid robot. This research involves numerous insights gleaned from cognitive psychology (drawn from both new and old theories of behavior) and integrating these techniques into QRIO's architecture with the long-term goal of providing highly satisfying long-term interaction and attachment formation by a human partner. The underlying models used and the results obtained on QRIO are presented.

Learning the Foundations for Life-Long Learning
Benjamin Kuipers (Univ. Texas-Austin)

Starting in early childhood, humans learn to conceptualize the world in terms of objects and actions, space and time, materials and affordances, and so on. The infant's world, described by William James as "blooming, buzzing confusion", is gradually replaced by a world of macroscopic things, relationships among them, and even agents communicating in language.

Developmental robotics explores how we can model this learning process computationally. In our laboratory, we investigate a simplified abstraction --- a robot starting with uninterpreted "pixel level" sensors and effectors --- and show how it can learn the structure of its sensorimotor system, control laws for moving among distinctive states, objects in the foreground individuated from the background, and actions that can be used to make and carry out plans.

Once the agent has learned initial versions of concepts of space, time, objects, actions, affordances, and so on, they serve as the foundations for learning more specific concepts such as block, box, grasp, put, dog, cat, move, and so on. Specific concepts are grounded in the foundational concepts, which in turn are grounded in sensorimotor interaction with the world.

With that foundation, the agent's knowledge representation gains adaptability and robustness against changes to the sensors and effectors. Having learned invariant properties of space, and the places and objects within it, those invariants can be used to detect and diagnose changes to the sensors and effectors. The grounding of foundational concepts, once established initially, can adapt to changed sensors and effectors.

This kind of adaptability is necessary but not sufficient. There is no single "silver bullet" for life-long learning. There are many different facets to learning about the world, to scaling up the knowledge representation, and to keeping learned knowledge current as the world and the agent's own sensorimotor system change.

Manipulation in Human Environments
Charlie Kemp (Georgia Tech)

The efficient acquisition and generalization of skills for manual tasks requires that a robot be able to perceive and control the important aspects of an object while ignoring irrelevant factors. For many tasks involving everyday tool-like objects, detection and control of the distal end of the object is sufficient for its use. For example, a robot could pour a substance from a bottle by controlling the position and orientation of the mouth. Likewise, the canonical tasks associated with a screwdriver, hammer, or pen rely on control of the tool’s tip. In this paper, we present methods that allow a robot to autonomously detect and control the tip of a tool-like object. We also show results for modeling the appearance of this important type of task relevant feature.

Fusing Robot Behaviors for Human-Level Tasks
Monica Nicolescu (Univ. Nevada-Reno)
Chad Jenkins (Brown Univ.)
Austin Stanhope (Univ. Nevada-Reno)

Behavior-based control is one of the most widely used approaches for autonomous robot control. However, in many robot systems, there is often a disconnect between a user’s desired task-level behavior and a robot’s preprogrammed (innate) capabilities. Typically, the space of robot behavior is limited to sequential performances, switching between the robot’s available skills. Such limited expression does not necessarily overlap with the space of desired robot behavior, leaving users unable to express their true desired control policy to the robot. To bridge this divide, a new approach is proposed, which integrates state estimation (as a particle filter), learning by demonstration, and behavior-based control into an approach for robot learning. While these methods have typically been used in different contexts, we demonstrate the ability to use state estimation in order to learn a user’s intended control policy from demonstration as a linear combination of innate behaviors. Through a specific navigation task, this method demonstrates how the same task-level behavior can be learned with different combinations of innate behaviors.

Toward Real-time Human Detection and Tracking in Diverse
Environments
Nate Koenig (USC, iRobot Corp.)

Human-robot interaction (HRI) encompasses numerous disciplines from psychology to mechanical engineering. Each field contributes to a robot’s ability to better understand and respond to humans actions, and behaviors. A common thread is the need to detect a person and their location within an
environment. In this paper we tackle the issue of identifying which objects in a scene are people. We also discuss current work on human detection, and strategies for improved performance.

Robot Learning via Socially Guided Exploration
Andrea Lockerd Thomaz (MIT)
Cynthia Breazeal (MIT)

We present a learning mechanism, Socially Guided Exploration, in which a robot learns new tasks through a combination of self-exploration and social interaction. The system’s motivational drives (novelty, mastery), along with social scaffolding from a human partner, bias behavior to create learning opportunities for a Reinforcement Learning mechanism. The system is able to learn on its own, but can flexibly use the guidance of a human partner to improve performance. An initial experiment shows how a human shapes the learning process through suggesting actions, drawing attention to goal states, and arranging the environment to encourage generalization

Towards Life-Long Learning in Household Robots: the Piagetian Approach
Michael Pardowitz
Rüdiger Dillmann

Learning is a core feature of future household robot systems. Nonetheless, present-day learning approaches fail to take into account that learning is never a finished process but an everyday task for biological systems. Additionally, humans always learn a various number of different tasks at the same time.

This paper proposes an approach to these two problems by applying the concept of Piagetian learning to the problem of robot task learning. It proposes a method for the autonomous recognition of different task classes in the robots experiences and gives one possibility, how this task knowledge can be exploited for incremental learning of sequential reordering features of a task. This framework is evaluated using three different tasks from the household domain.

Learning Elements of Robot Soccer from Demonstration
Dan Grollman (Brown University)
Chad Jenkins (Brown University)

We seek to enable users to teach personal robots arbitrary tasks, allowing robots to better serve users’ wants and needs without explicit programming. Robot learning from demonstration is an approach well-suited to this paradigm, as a robot learns new tasks in new environments from observations of the task itself. Many current robot learning algorithms require the existence of basic behaviors that can be combined to perform the desired task. However, robots that exist in the world for long timeframes may exhaust this basis set. In particular, a robot may be asked to perform an unknown task for which its built in behaviors may not be appropriate. We demonstrate a learning framework that is capable of learning both low-level motion primitives (locomotion and manipulation) and high-level tasks built on top of them from interactive demonstration. We apply nonparametric regression within this framework towards learning a complete robot soccer player and successfully teach a robot dog to first walk, and then to seek and acquire a ball.