CMU talk: Learning to Search: Structured Prediction Techniques for Imitation Learning

PhD Thesis Defense:

Learning to Search: Structured Prediction Techniques for Imitation Learning

Nathan D. Ratliff
Carnegie Mellon University

May 01, 2009

Abstract: Modern robots successfully manipulate objects, navigate rugged terrain, drive in urban settings, and play world-class chess. Unfortunately, programming these robots is challenging, time-consuming and expensive; the parameters governing their behavior are often unintuitive, even when the desired behavior is clear and easily demonstrated. Inspired by successful end-to-end learning systems such as neural network controlled driving platforms (Pomerleau, 1989), learning-based "programming by demonstration" has gained currency as a method to achieve intelligent robot behavior. Unfortunately, with highly structured algorithms at their core, it is not clear how to effectively and efficiently train modern robotic systems using classical learning techniques. Rather than redefining robot architectures to accommodate existing learning algorithms, in this thesis I develop learning techniques that leverage the performance of modern robotic components.

My presentation begins with a discussion of a novel imitation learning framework we call Maximum Margin Planning which automates finding a cost function for optimal planning and control algorithms such as A*. In the linear setting, this framework has firm theoretical backing in the form of strong generalization and regret bounds. Further, I have developed practical nonlinear generalizations that are effective and efficient for real-world problems. This framework reduces imitation learning to a modern form of machine learning known as Maximum Margin Structured Classification (Taskar et al. 2005); these algorithms, therefore, apply both specifically to training existing state-of-the-art planners, as well as broadly to solving a range of structured prediction problems of importance in learning and robotics.

In difficult high-dimensional planning domains, such as those found in many manipulation problems, high-performance planning technology remains a topic of much research. I will present some recent work which moves toward simultaneously advancing this technology while retaining the learnability developed above.

I'll demonstrate our algorithms on a range of applications including overhead navigation, quadrupedal locomotion, heuristic learning, manipulation planning, grasp prediction, driver prediction, pedestrian prediction, optical character recognition, and LADAR classification.

link