Thesis talk: Seeing the World Behind the Image

Date: 10 July 2007
Time: 1:00 p.m.
Place: Newell Simon Hall 1305
Type: Thesis Oral
Who: Derek Hoiem
Topic: Seeing the World Behind the Image: Spatial Layout for 3D Scene
Understanding

Abstract:
When humans look at an image, they see not just a pattern of color and
texture, but the world behind the image. In the same way, computer
vision algorithms must go beyond the pixels and reason about the
underlying scene. In this dissertation, we propose methods to recover
the basic spatial layout from a single image and begin to investigate
its use as a foundation for scene understanding.

Our spatial layout is a description of the 3D scene in terms of
surfaces, occlusions, camera viewpoint, and objects. We propose a
geometric class representation, a coarse categorization of surfaces
according to their 3D orientations, and learn appearance-based models of
geometry to identify surfaces in an image. These surface estimates serve
as a basis for recovering the boundaries and occlusion relationships of
prominent objects. We further show that simple reasoning about camera
viewpoint and object size in the image allows accurate inference of the
viewpoint and greatly improves object detection. Finally, we demonstrate
the potential usefulness of our methods in applications to 3D
reconstruction, scene synthesis, and robot navigation.

Thesis Committee Members:
Alexei A. Efros, Co-Chair
Martial Hebert, Co-Chair
Takeo Kanade
Rahul Sukthankar, Intel Research Pittsburgh
William T. Freeman, Massachusetts Institute of Technology

A draft of the thesis document is available at:
link