[Lab Meeting] May 5th, 2008 (Hero): An Application of Reinforcement Learning to Aerobatic Helicopter Flight

Pieter Abbeel, Adam Coates, Morgan Quigley, Andrew Y. Ng
Computer Science Dept.
Stanford University
Stanford, CA 94305

Abstract
Autonomous helicopter flight is widely regarded to be a highly challenging control
problem. This paper presents the first successful autonomous completion on a
real RC helicopter of the following four aerobatic maneuvers: forward flip and
sideways roll at low speed, tail-in funnel, and nose-in funnel. Our experimental
results significantly extend the state of the art in autonomous helicopter flight.
We used the following approach: First we had a pilot fly the helicopter to help
us find a helicopter dynamics model and a reward (cost) function. Then we used
a reinforcement learning (optimal control) algorithm to find a controller that is
optimized for the resulting model and reward function. More specifically, we used
differential dynamic programming (DDP), an extension of the linear quadratic
regulator (LQR).

link: http://www.cs.stanford.edu/%7Epabbeel/pubs/AbbeelCoatesQuigleyNg_aaorltahf_nips2006.pdf