[SCS Faculty Candidate Talk] Analyzing Dynamic Scenes from Moving Cameras: A Spacetime

SCS Faculty Candidate Yaser Sheikh
Wednesday, May 7th, 2008
10:00 a.m. NSH 3305
Host: Srinivasa Narasimhan, The Robotics Institute

Title:* *Analyzing Dynamic Scenes from Moving Cameras: A Spacetime Perspective

Abstract:
With the proliferation of camera-enabled cell phones, domestic robots, and wearable computers, moving cameras are being introduced /en masse/into society. The confluence of camera motion and the motion of objects in the scene complicates the task of understanding the scene from video. In this talk, I discuss how and when it is possible to disambiguatet hese two sources of motion, towards the goal of analyzing dynamic scenes from moving cameras.

I begin by considering a single camera viewing a dynamic scene. Unlike contemporary approaches to this problem, which try to model the variation in the shape of objects, I show that modeling the variation of points along time is better motivated physically and produces more stable reconstructions. This model also intuitively characterizes the inherent reconstruction ambiguity for a single camera and motivates the study of dynamic scenes from /multiple/ moving cameras. I present the case for conducting this analysis in spacetime, where a dynamic scene is considered a body in spacetime, and each video a spacetime image of this body. Through this representation, I demonstrate that classic algorithmsin multiview geometry that deal with static scenes can be lifted to spacetime, and applied directly for dynamic scene analysis. The analogues of factorization approaches and the fundamental matrix are described, leading to new, intuitive, relationships between the epipolar geometries of perspective images, linear pushbroom images and epipolar plane images.