Lab Meeting, Jun 19, 2014 (Jim): Bayesian Exploration and Interactive Demonstration in Continuous State MAXQ-Learning

Title:
Bayesian Exploration and Interactive Demonstration in Continuous State MAXQ-Learning

IEEE International Conference on Robotics and Automation, May, 2014.

Author:
Kathrin Gräve and Sven Behnke

Abstract:
... Inspired by the way humans decompose complex tasks, hierarchical methods for robot learning have attracted significant interest. In this paper, we apply the MAXQ method for hierarchical reinforcement learning to continuous state spaces. By using Gaussian Process Regression for MAXQ value function decomposition, we obtain probabilistic estimates of primitive and completion values for every subtask within the MAXQ hierarchy. ... Based on the expected deviation of these estimates, we devise a Bayesian exploration strategy that balances optimization of expected values and risk from exploring unknown actions. To further reduce risk and to accelerate learning, we complement MAXQ with learning from demonstrations in an interactive way. In every situation and subtask, the system may ask for a demonstration if there is not enough knowledge available to determine a safe action for exploration. We demonstrate the ability of the proposed system to efficiently learn solutions to complex tasks on a box stacking scenario.

Link