CMU talk: Observing and Interpreting Everyday Human Activities

VASC Seminar
Monday, March 30, 2009

Observing and Interpreting Everyday Human Activities
Jan Bandouch, Ph.D. Student
Moritz Tenorth, Ph.D. Student
TU Munich, Germany

Abstract:
We will present and discuss an integrated approach for the automated perception, interpretation and analysis of human activities of daily life, with an emphasis on everyday manipulation tasks. During the first half of this talk, we will describe a markerless motion capture system that we use to acquire joint-angle representations of human full-body motions at high accuracy. The system is capable of tracking arbitrary, previously unobserved motions using a sophisticated hierarchical sampling strategy for recursive Bayesian estimation that combines partitioning with annealing strategies to enable efficient search in the presence of many local maxima. A simple yet effective appearance model is used to implicitly deal with occlusions and to reduce the influence of objects and dynamic parts of the environment. We will then show in the second half of our talk how to infer probabilistic, hybrid (continuous/discrete), low-dimensional, and hierarchical models of the observed activity. These models can be used to answer queries about the observed activity such as the following ones: Which activity was performed? Which hand trajectory did the human use for taking a plate out of the overhead cupboard? What was different compared to the normal execution of this activity? The integration of these activity models into a knowledge-based framework allows for the association of the observed activity with encyclopedic, commonsense, and naive physics knowledge and for querying the system in an abstract, symbolic way.

Bio:
Jan Bandouch is a PhD student in the Intelligent Autonomous Systems Group of Prof. Michael Beetz at the TU Munich, Germany. His current research interest is in Computer Vision and Robotics, where he is working on techniques for markerless human motion capture in typical human living environments. The intended areas of application are activity and intention recognition in smart environments, motion analysis and ergonomic studies. He obtained his Diploma (equivalent to M.Sc.) in Computer Science from the TU Munich in 2005. Homepage: http://www9.cs.tum.edu/people/bandouch

Moritz Tenorth is a PhD student in the Intelligent Autonomous Systems Group of Prof. Michael Beetz at the TU Munich, Germany. His research interests include grounded knowledge representations which integrate information from web sources, observed sensor data and data mining techniques, and their applications to knowledge-based action interpretation and robot control. He studied Electical Engineering in Aachen and Paris and obtained his Diploma degree (equivalent to M.Eng.) in 2007 from the RWTH Aachen. Homepage: http://www9.cs.tum.edu/people/tenorth