[CVPR 2008] A Mobile Vision System for Robust Multi-Person Tracking

Authors: Andreas Ess, Bastian Leibe, Konrad Schindler and Luc Van Gool
ETH Zurich, Switzerland, KU Leuven, Belgium
CVPR 2008 oral

Full text

Abstract:
We present a mobile vision system for multi-person track-ing in busy environments. Specifically, the system integratescontinuous visual odometry computation with tracking-by-detection in order to track pedestrians in spite of frequentocclusions and egomotion of the camera rig. To achieve re-liable performance under real-world conditions, it has longbeen advocated to extract and combine as much visual in-formation as possible. We propose a way to closely inte-grate the vision modules for visual odometry, pedestrian de-tection, depth estimation, and tracking. The integration nat-urally leads to several cognitive feedback loops between themodules. Among others, we propose a novel feedback con-nection from the object detector to visual odometry whichutilizes the semantic knowledge of detection to stabilize lo-calization. Feedback loops always carry the danger that er-roneous feedback from one module is amplified and causesthe entire system to become instable. We therefore incor-porate automatic failure detection and recovery, allowingthe system to continue when a module becomes unreliable.The approach is experimentally evaluated on several longand difficult video sequences from busy inner-city locations.Our results show that the proposed integration makes it pos-sible to deliver stable tracking performance in scenes ofpreviously infeasible complexity.