CMU Thesis Proposal: Geometrically Coherent Image Interpretation

Derek Hoiem
Robotics Institute, Carnegie Mellon University

Abstract: Objects in the world interact and are constrained according to their 3D geometry. Thus, inference of 3D geometry provides a natural interface for relating objects and performing actions such as navigation. We propose a simple class-based representation for 3D geometric information, in which we estimate 3D orientations from a single image using appearance-based models. With knowledge the scene's geometry, we can improve image understanding algorithms, including object detection, material labeling, and scene recognition. We focus on improving object detection using our geometric context and estimates of the camera parameters. Rather than simply using estimated geometry as features into a subsequent classification system, however, we propose to determine a coherent hypothesis that enforces the strong geometric relationships among the individual object and surface hypotheses. We develop a probabilistic formulation for encoding the relations of different types of scene information and describe inference algorithms for posing queries about the scene.

Further Details: A copy of the thesis proposal document can be found at http://www.cs.cmu.edu/~dhoiem/hoiemproposal.pdf.