talk at CMU: Tracking Across Multiple Moving Cameras

Dr. Mubarak Shah
Computer Vision Lab, School of Computer Science
University of Central Florida, Orlando, FL 32816
http://www.cs.ucf.edu/~vision/

Check out their ICCV2005 papers!

The concept of a cooperative multi-camera system, informally a 'forest' of sensors, has recently received increasing attention from the research community. The idea is of great practical relevance, since cameras typically have limited fields of view, but are now available at low costs. Thus, instead of having a high-resolution camera that surveys a large area, far greater flexibility and scalability can be achieved by observing a scene 'through many eyes', using a multitude of lower-resolution COTS (commercial off-the-shelf) cameras.

In this talk I will present two approaches for object tracking across multiple moving cameras. In the first approach, objects are to be tracked across several cameras, each mounted on an aerial vehicle, without any telemetry or calibration information. The principal assumption that is made in this work is that the altitude of the camera allows the scene to be modeled well by a plane. First the global motion is compensated in each video sequence and objects are detected and tracked in individual cameras. For solving multiple camera correspondence problem we exploit constraints on the relationship between the motion of each object across cameras, estimating the probability that trajectories in two views originated from the same object, to test multiple correspondence hypotheses (without assuming any calibration information).

In the second approach we consider sequences acquired by hand-held cameras, for which planar scene assumption is not valid. Recently we have proposed a notion of temporal fundamental matrix to capture the epi-polar geometry between the temporal views of independently moving camera pair where the scene is dynamic. The temporal fundamental matrix, which is a 3x3 matrix capturing the temporal variation of the geometry. Constraining the rotational and translational motion of cameras to polynomials in time, we have shown that the components of the fundamental matrix are polynomials in time. In order to obtain the correct correspondences across the multiple moving cameras, we perform a maximum bipartite matching of a graph, in which the weights of the edges depend on the properties of the temporal fundamental matrix.

======================================
Dr. Mubarak Shah, Agere Chair professor of Computer Science, and the founding director of the Computer Vision Laboratory at University of Central Florida (UCF), is a researcher in computer vision. He is a co-author of two books Video Registration (2003) and Motion-Based Recognition (1997), both by Kluwer Academic Publishers. He has worked in several areas including activity and gesture recognition, violence detection, event ontology, object tracking (fixed camera, moving camera, multiple overlapping and non-overlapping cameras), video segmentation, story and scene segmentation, view morphing, ATR, wide-baseline matching, and video registration. . Dr. Shah is a fellow of IEEE, was an IEEE Distinguished Visitor speaker for 1997-2000, and is often invited to present seminars, tutorials and invited talks all over the world. He received the Harris Corporation Engineering Achievement Award in 1999, the TOKTEN awards from UNDP in 1995, 1997, and 2000; Teaching Incentive Program award in 1995 and 2003, Research Incentive Award in 2003, and IEEE Outstanding Engineering Educator Award in 1997. He is an editor of international book series on "Video Computing"; editor in chief of Machine Vision and Applications journal, and an associate editor Pattern Recognition journal. He was an associate editor of the IEEE Transactions on PAMI, and a guest editor of the special issue of International Journal of Computer Vision on Video Computing.