Human Pose and Action Recognition for Interactive Robots
There is currently a division between real-world human performance and the decision making of socially interactive robots. Specifically, the decision making of robots needs to have information about the decision making of its human collaborators. This circumstance is partially due to the difficulty in estimating human cues, such as pose and gesture, from robot sensing. Towards crossing this division, we present a method for kinematic pose estimation and action recognition from monocular robot vision through the use of dynamical human motion vocabularies. Our notion of motion vocabulary is comprised of primitives that structure a human's action space for decision making and predict human movement dynamics. Through prediction, such primitives can be used to both generate motor commands for specific actions and perceive humans performing those actions. In this paper, we focus specifically on the perception of human pose and performed actions using a known vocabulary of motion primitives. Given image observations over time, each primitive infers pose independently using its expected dynamics in the context of a particle filter. Pose estimates from a set of primitives inferencing in parallel are arbitrated to estimate the action being performed. The efficacy of our approach is demonstrated through interactive-time pose and action recognition over extended motion trials. Results evidence our approach requires small numbers of particles for tracking, is robust to unsegmented multi-action movement, movement speed, camera viewpoint and is able to recover from occulsions.
O. Jenkins, G. González, and M. Loper. Interactive Human Pose and Action Recognition using Dynamical Motion Primitives. In International Journal of Humanoid Robotics, in press, 2007.
O. Jenkins, G. González, and M. Loper. Tracking human motion and actions for interactive robots. In Human-Robot Interaction (HRI 2007), Arlington, VA, USA, Mar 2007.
[ bib | .pdf ]
O. Jenkins, G. González, and M. Loper. Learning dynamical motion vocabularies for kinematic tracking and activity recognition. In CVPR 2006 Workshop on Vision for Human-Computer Interaction, New York, NY, USA, Jun 2006.
[ bib | http ]
O. Jenkins, G. González, and M. Loper. Monocular virtual trajectory estimation with dynamical primitives. In AAAI 2006 Cognitive Robotics Workshop, Boston, MA, USA, Jul 2006.
[ bib | .pdf ]
Office of Naval Research, Award N000140710141 DARPA ``Tactical Teams'' SBIR (with iRobot Corp.) ONR ``Natural Human-Robot Interaction'' SBIR (with iRobot Corp.)