Lab meeting July 25th 2013 (Benny): Learning to segment and track in RGBD

Presented by: Benny

From: IEEE Transactions on Automation Science and Engineering 2013

Authors: Alex Teichman and Jake Lussier and Sebastian Thrun

Link: Paper

Abstract: We consider the problem of segmenting and tracking deformable objects in color video with depth (RGBD) data available from commodity sensors such as the Asus Xtion Pro Live or Microsoft Kinect. We frame this problem with very few assumptions - no prior object model, no stationary sensor, no prior 3D map - thus making a solution potentially useful for a large number of applications, including semi-supervised learning, 3D model capture, and object recognition.

Our approach makes use of a rich feature set, including local image appearance, depth discontinuities, optical flow, and surface normals to inform the segmentation decision in a conditional random field model. In contrast to previous work in this field, the proposed method learns how to best make use of these features from ground-truth segmented sequences. We provide qualitative and quantitative analyses which demonstrate substantial improvement over the state of the art.

This paper is an extended version of our previous work [29]. Building on this, we show that it is possible to achieve an order of magnitude speedup and thus real-time performance ( 20FPS) on a laptop computer by applying simple algorithmic optimizations to the original work. This speedup comes at only a minor cost in

overall accuracy and thus makes this approach applicable to a broader range of tasks. We demonstrate one such task: real-time, online, interactive segmentation to efficiently collect training data for an off-the-shelf object detector.

Lab Meeting July 10th, 2013 (Andi): Probabilistic Models for 3D Urban Scene Understanding from Movable Platforms

Title: Probabilistic Models for 3D Urban Scene Understanding from Movable Platforms

PhD Thesis, Andreas Geiger (KIT)

Abstract:

Visual 3D scene understanding is an important component in autonomous driving and robot navigation. Intelligent vehicles for example often base their decisions on observations obtained from video cameras as they are cheap and easy to employ. Inner-city intersections represent an interesting but also very challenging scenario in this context: The road layout may be very complex and observations are often noisy or even missing due to heavy occlusions. While Highway navigation (e.g., Dickmanns et al. [49]) and autonomous driving on simple and annotated intersections (e.g., DARPA Urban Challenge [30]) have already been demonstrated successfully, understanding and navigating general inner-city crossings with little prior knowledge remains an unsolved problem. This thesis is a contribution to understanding multi-object traffic scenes from video sequences. All data is provided by a camera system which is mounted on top of the autonomous driving platform AnnieWAY [103]. The proposed probabilistic generative model reasons jointly about the 3D scene layout as well as the 3D location and orientation of objects in the scene. In particular, the scene topology, geometry as well as traffic activities are inferred from short video sequences. The model takes advantage of monocular information in the form of vehicle tracklets, vanishing lines and semantic labels. Additionally, the benefit of stereo features such as 3D scene flow and occupancy grids is investigated.

Motivated by the impressive driving capabilities of humans, no further information such as GPS, lidar, radar or map knowledge is required. Experiments conducted on 113 representative intersection sequences show that the developed approach successfully infers the correct layout in a variety of difficult scenarios. To evaluate the importance of each feature cue, experiments with different feature combinations are conducted. Additionally, the proposed method is shown to improve object detection and object orientation estimation performance.

based primarily on the following two papers: (CVPR + NIPS '11)
http://ttic.uchicago.edu/~rurtasun/publications/geiger_etal_cvpr11.pdf
http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2011_0842.pdf

Lab Meeting July 3rd, 2013 (Jeff): Switchable Constraints vs. Max-Mixture Models vs. RRR - A Comparison of Three Approaches to Robust Pose Graph SLAM

Title: Switchable Constraints vs. Max-Mixture Models vs. RRR - A Comparison of Three Approaches to Robust Pose Graph SLAM

Authors: Niko Sünderhauf and Peter Protzel

Abstract:

SLAM algorithms that can infer a correct map despite the presence of outliers have recently attracted increasing attention. In the context of SLAM, outlier constraints are typically caused by a failed place recognition due to perceptional aliasing. If not handled correctly, they can have catastrophic effects on the inferred map. Since robust robotic mapping and SLAM are among the key requirements for autonomous long-term operation, inference methods that can cope with such data association failures are a hot topic in current research. Our paper compares three very recently published approaches to robust pose graph SLAM, namely switchable constraints, max-mixture models and the RRR algorithm. All three methods were developed as extensions to existing factor graph-based SLAM back-ends and aim at improving the overall system’s robustness to false positive loop closure constraints. Due to the novelty of the three proposed algorithms, no direct comparison has been conducted so far.


IEEE International Conference on Robotics and Automation (ICRA), 2013

Link:
LocalLink
http://www.tu-chemnitz.de/etit/proaut/rsrc/ICRA12-comparisonRobustSLAM.pdf

Reference Link:
Switchable Constraints
http://www.tu-chemnitz.de/etit/proaut/mitarbeiter/rsrc/IROS12-switchableConstraints.pdf
Max-Mixture
http://www.roboticsproceedings.org/rss08/p40.pdf
RRR
http://www.roboticsproceedings.org/rss08/p30.pdf

Lab Meeting Jun. 19, 2013 (Alan) : Dense Variational Reconstruction of Non-Rigid Surfaces from Monocular Video

Title: Dense Variational Reconstruction of Non-Rigid Surfaces from Monocular Video (CVPR 2013 Oral)
Authors: Ravi Garg, Anastasios Roussos, Lourdes Agapito

Abstract
This paper offers the first variational approach to the problem of dense 3D reconstruction of non-rigid surfaces from a monocular video sequence. We formulate nonrigid structure from motion (NRSfM) as a global variational energy minimization problem to estimate dense low-rank smooth 3D shapes for every frame along with the camera motion matrices, given dense 2D correspondences.
Unlike traditional factorization based approaches to NRSfM, which model the low-rank non-rigid shape using a fixed number of basis shapes and corresponding coefficients, we minimize the rank of the matrix of time-varying shapes directly via trace norm minimization. In conjunction with this low-rank constraint, we use an edge preserving total-variation regularization term to obtain spatially smooth shapes for every frame. Thanks to proximal splitting techniques the optimization problem can be decomposed into many point-wise sub-problems and simple linear systems which can be easily solved on GPU hardware. We show results on real sequences of different objects (face, torso, beating heart) where, despite challenges in tracking, illumination changes and occlusions, our method reconstructs highly deforming smooth surfaces densely and accurately directly from video, without the need for any prior models or shape templates.

Link

Lab meeting May 29th 2013 (Jim): Reciprocal collision avoidance

I'm going to present the idea of "reciprocal collision avoidance": each moving agent
should take responsibilities for collision avoidance with each other during the navigation. Based on the model of velocity obstacles, the "reciprocal velocity obstacles" and its variations are developed for multi-agent navigation. The main references / materials are the following papers:

Reciprocal Velocity Obstacles for Real-time Multi-agent Navigation
Jur van den Berg, Ming C. Lin, Dinesh Manocha
IEEE International Conference on Robotics and Automation (ICRA), 2008
website

TheHybrid Reciprocal Velocity Obstacle
Jamie Snape, Jur van den Berg, Stephen J. Guy, Dinesh Manocha
IEEE Transactions on Robotics (T-RO), vol. 27, pp. 696-706, 2011

website

Reciprocaln-body Collision Avoidance
Jur van den Berg, Stephen J. Guy, Ming C. Lin, Dinesh Manocha
Robotics Research: The 14th International Symposium (ISRR), Springer Tracts in Advanced Robotics (STAR), vol. 70, pp. 3-19, 2011

website

Lab meeting May 22th 2013 (Tom Hsu): Incorporating User Interaction and Topological Constraints within Contour Completion via Discrete Calculus

Presented by: Tom Hsu

From: Proc. of the Computer Vision and Pattern Recognition (CVPR'13),  Portland, Oregon 2013.

Authors: Jia Xu Maxwell D. Collins Vikas Singh (University of Wisconsin-Madison)

Link: Paper

Abstract: 

We study the problem of interactive segmentation and contour completion for multiple objects. The form of constraints our model incorporates are those coming from user scribbles (interior or exterior constraints) as well as information regarding the topology of the 2-D space after partitioning (number of closed contours desired). We discuss how concepts from discrete calculus and a simple identity using the Euler characteristic of a planar graph can be utilized to derive a practical algorithm for this problem. We also present specialized branch and bound methods for the case of single contour completion under such constraints. On an extensive dataset of ~1000 images, our experiments suggest that a small amount of side knowledge can give strong improvements over fully unsupervised contour completion methods. We show that by interpreting user indications topologically, user effort is substantially reduced.

Lab meeting Mar 8th 2013 (Gene): Lost! Leveraging the Crowd for Probabilistic Visual Self-Localization

Presented by: Gene

From: CVPR2013

Authors: Marcus A. Brubaker, Andreas Geiger, Raquel Urtasun

Abstract:

In this paper we propose an affordable solution to selflocalization, which utilizes visual odometry and road maps as the only inputs. To this end, we present a probabilistic model as well as an efficient approximate inference algorithm, which is able to utilize distributed computation to meet the real-time requirements of autonomous systems. Because of the probabilistic nature of the model we are able to cope with uncertainty due to noisy visual odometry and inherent ambiguities in the map (e.g., in a Manhattan world). By exploiting freely available, community developed maps and visual odometry measurements, we are able to localize a vehicle up to 3m after only a few seconds of driving on maps which contain more than 2,150km of drivable roads.


Link

Lab meeting Apr 24th 2013 (Hank Lin): Scene Parsing with Multiscale Feature Learning, Purity Trees, and Optimal Covers

Presented by: Hank Lin

From: Proc. of the International Conference on Machine Learning (ICML'12), Edinburgh, Scotland, 2012.

Authors: C. Farabet, C. Couprie, L. Najman, Y. LeCun

Link: Paper Video

Abstract: 

     Scene parsing, or semantic segmentation, consists in la-

beling each pixel in an image with the category of the object

it belongs to. It is a challenging task that involves the simul-

taneous detection, segmentation and recognition of all the

objects in the image.

     The scene parsing method proposed here starts by com-

puting a tree of segments from a graph of pixel dissimilari-

ties. Simultaneously, a set of dense feature vectors is com-

puted which encodes regions of multiple sizes centered on

each pixel. The feature extractor is a multiscale convolu-

tional network trained from raw pixels. The feature vec-

tors associated with the segments covered by each node in

the tree are aggregated and fed to a classifier which pro-

duces an estimate of the distribution of object categories

contained in the segment. A subset of tree nodes that cover

the image are then selected so as to maximize the aver-

age “purity” of the class distributions, hence maximizing

the overall likelihood that each segment will contain a sin-

gle object. The convolutional network feature extractor is

trained end-to-end from raw pixels, alleviating the need for

engineered features. After training, the system is parameter

free.

      The system yields record accuracies on the Stanford

Background Dataset (8 classes), the Sift Flow Dataset (33

classes) and the Barcelona Dataset (170 classes) while

being an order of magnitude faster than competing ap-

proaches, producing a 320 × 240 image labeling in less

than 1 second.

Lab meeting Apr 17th 2013 (Bang-Cheng Wang): Biped Walking Pattern Generation by using Preview Control of Zero-Moment Point

Presented by Bang-Cheng Wang

From Proceedings of the 2003 IEEE
International Conference on Robotics & Automation
Taipei, Taiwan, September 14-19, 2003.

Authors:
Shuuji KAJITA, Fumio KANEHIRO, Kenji KANEKO, Kiyoshi FUJIWARA,
Kensuke HARADA, Kazuhito YOKOI and Hirohisa HIRUKAWA

Abstract:
We introduce a new method of a biped walking pattern
generation by using a preview control of the zero moment
point (ZMP). First, the dynamics of a biped
robot is modeled as a running cart on a table which
gives a convenient representation to treat ZMP. After
reviewing conventional methods of ZMP based pattern
generation, we formalize the problem as the design of a
ZMP tracking servo controller. It is shown that we can
realize such controller by adopting the preview control
theory that uses the future reference. It is also shown
that a preview controller can be used to compensate
the ZMP error caused by the difference between a simple
model and the precise multibody model. The effectiveness
of the proposed method is demonstrated by a
simulation of walking on spiral stairs.

Link

Lab Meeting April 10, 2013 (Jimmy): Geodesic Flow Kernel for Unsupervised Domain Adaptation

Title: Geodesic Flow Kernel for Unsupervised Domain Adaptation
Authors: Boqing Gong, Yuan Shi, Fei Sha, Kristen Grauman
In: CVPR2012

Abstract
In real-world applications of visual recognition, many factors—such as pose, illumination, or image quality—can cause a significant mismatch between the source domain on which classifiers are trained and the target domain to which those classifiers are applied. As such, the classifiers often perform poorly on the target domain. Domain adaptation techniques aim to correct the mismatch. Existing approaches have concentrated on learning feature representations that are invariant across domains, and they often do not directly exploit low-dimensional structures that are intrinsic to many vision datasets. In this paper, we propose a new kernel-based method that takes advantage of such structures. Our geodesic flow kernel models domain shift by integrating an infinite number of subspaces that characterize changes in geometric and statistical properties from the source to the target domain. Our approach is computationally advantageous, automatically inferring important algorithmic parameters without requiring extensive crossvalidation or labeled data from either domain. We also introduce a metric that reliably measures the adaptability between a pair of source and target domains. For a given target domain and several source domains, the metric can be used to automatically select the optimal source domain to adapt and avoid less desirable ones. Empirical studies on standard datasets demonstrate the advantages of our approach over competing methods.

[link]

Lab Meeting, March 28, 2013 (Chiang Yi): Efficient Model-based 3D Tracking of Hand Articulations using Kinect (BMVC 2011)

Authors: Iason Oikonomidis, Nikolaos Kyriazis
 ,Antonis A. Argyros

Abstract:
We present a novel solution to the problem of recovering and tracking the 3D po-
sition, orientation and full articulation of a human hand from markerless visual obser-
vations obtained by a Kinect sensor. We treat this as an optimization problem, seeking
for the hand model parameters that minimize the discrepancy between the appearance
and 3D structure of hypothesized instances of a hand model and actual hand observa-
tions. This optimization problem is effectively solved using a variant of Particle Swarm
Optimization (PSO). The proposed method does not require special markers and/or a
complex image acquisition setup. Being model based, it provides continuous solutions
to the problem of tracking hand articulations. Extensive experiments with a prototype
GPU-based implementation of the proposed method demonstrate that accurate and ro-
bust 3D tracking of hand articulations can be achieved in near real-time (15Hz).


LINK

extended work: Tracking the articulated motion of two strongly interacting hands

Lab Meeting, March 21, 2013 (Yen-Ting): Extracting 3D Scene-Consistent Object Proposals and Depth from Stereo Images (ECCV 2012)

Authors: Michael Bleyer, Christoph Rhemann, and Carsten Rother

Abstract: This work combines two active areas of research in computer vision: unsupervised object extraction from a single image, and depth estimation from a stereo image pair. A recent, successful trend in unsupervised object extraction is to exploit so-called “3D scene-consistency”, that is enforcing that objects obey underlying physical constraints of the 3D scene, such as occupancy of 3D space and gravity of objects. Our main contribution is to introduce the concept of 3D scene-consistency into stereo matching. We show that this concept is beneficial for both tasks, object extraction and depth estimation. In particular, we demonstrate that our approach is able to create a large set of 3D scene-consistent object proposals, by varying e.g. the prior on the number of objects...

Link

Lab Meeting, March 14, 2013 (Channing): The Design of LEO: a 2D Bipedal Walking Robot for Online Autonomous Reinforcement Learning (IROS 2010)

Authors: Erik Schuitema, Martijn Wisse, Thijs Ramakers and Pieter Jonker

Abstract: Real robots demonstrating online Reinforcement Learning (RL) to learn new tasks are hard to find. The specific properties and limitations of real robots have a large impact on their suitability for RL experiments. In this work, we derive the main hardware and software requirements that a RL robot should fulfill, and present our biped robot LEO that was specifically designed to meet these requirements. We verify its aptitude in autonomous walking experiments using a pre-programmed controller. Although there is room
for improvement in the design, the robot was able to walk, fall and stand up without human intervention for 8 hours, during which it made over 43,000 footsteps.

Link

Lab Meeting, March 7, 2013 (Benny):A Segmentation and Data Association Annotation System for Laser-based Multi-Target Tracking Evaluation

Author: Chien-Chen Weng, Chieh-Chih Wang and Jennifer Healey

Abstract—2D laser scanners are now widely used to accomplish robot perception tasks such as SLAM and multi-target tracking (MTT). While a number of SLAM benchmarking datasets are available, only a few works have discussed the issues of collecting multi-target tracking benchmarking datasets.

In this work, a segmentation and data association annotation system is proposed for evaluating multi-target tracking using 2D laser scanners. The proposed annotation system uses the existing MTT algorithm to generate initial annotation results and uses camera images as the strong hints to assist annotators to recognize moving objects in laser scans. The annotators can draw the object’s shape and future trajectory to automate segmentation and data association and reduce the annotation task loading. The user study results show that the performance of the proposed annotation system is superior in the V-measure vs. annotation speed tests and the false positive and false negative rates.

Link

Lab meeting Feb. 21, 2013 (ChihChung) A Tensor-Based Algorithm for High-Order Graph Matching (PAMI 2010)

Authors: Olivier Duchenne, Francis Bach, In-So Kweon, and Jean Ponce

Abstract: This paper addresses the problem of establishing correspondences between two sets of visual features using higher-order constraints instead of the unary or pairwise ones used in classical methods. Concretely, the corresponding hypergraph matching problem is formulated as the maximization of a multi-linear objective function over all permutations of the features. This function is defined by a tensor representing the affinity between feature tuples. It is maximized using a generalization of spectral techniques where a relaxed problem is first solved by a multi-dimensional power method, and the solution is then projected onto the closest assignment matrix. The proposed approach has been implemented, and it is compared to state-of-the-art algorithms on both synthetic and real data.

Link

Lab meeting Jan. 23, 2013 (Gene): Fully Distributed Scalable Smoothing and Mapping with Robust Multi-robot Data Association (IEEE 2012)

Title: Fully Distributed Scalable Smoothing and Mapping with Robust Multi-robot Data Association (IEEE 2012)
Authors: Alexander Chunningham, Kai M. Wurm, Wolfarm Burgard, and Frank Dellaert

Abstract:


In this paper we focus on the multi-robot perception problem, and present an experimentally validated end-to-end multi-robot mapping framework, enabling individual robots in a team to see beyond their individual sensor horizons. The inference part of our system is the DDF-SAM algorithm [1], which provides a decentralized communication and inference scheme, but did not address the crucial issue of data association.

One key contribution is a novel, RANSAC-based, approach for performing the between-robot data associations and initialization of relative frames of reference. We demonstrate this system with both data collected from real robot experiments, as well as in a large scale simulated experiment demonstrating the scalability of the proposed approach.

Link

Lab meeting Jan 9th 2013 (Bang-Cheng Wang): Kicking a Ball – Modeling Complex Dynamic Motions for Humanoid Robots

Presented by Bang-Cheng Wang

From RoboCup 2010: Robot Soccer World Cup XIV, ser. Lecture Notes 

in Artificial Intelligence, E. Chown, A. Matsumoto, P. Pl¨oger, 

and J. R. del Solar, Eds. Springer, to appear in 2011.

Authors:
Judith Müller, Tim Laue, and Thomas Röfer

Abstract:
Complex motions like kicking a ball into the goal are becoming
more important in RoboCup leagues such as the Standard Platform
League. Thus, there is a need for motion sequences that can be parameterized
and changed dynamically. This paper presents a motion engine
that translates motions into joint angles by using trajectories. These
motions are defined as a set of Bezier curves that can be changed online
to allow adjusting, for example, a kicking motion precisely to the actual
position of the ball. During the execution, motions are stabilized by
the combination of center of mass balancing and a gyro feedback-based
closed-loop PID controller.

Lab Meeting December 19th, 2012 (Jeff): Inference on networks of mixtures for robust robot mapping

Title: Inference on networks of mixtures for robust robot mapping

Authors: Edwin Olson and Pratik Agarwal

Abstract:

The central challenge in robotic mapping is obtaining reliable data associations (or “loop closures”): state-of-the-art inference algorithms can fail catastrophically if even
one erroneous loop closure is incorporated into the map. Consequently, much work has been done to push error rates closer to zero. However, a long-lived or multi-robot system will
still encounter errors, leading to system failure.

We propose a fundamentally different approach: allow richer error models that allow the probability of a failure to be explicitly modeled. In other words, we optimize the map while
simultaneously determining which loop closures are correct from within a single integrated Bayesian framework. Unlike earlier multiple-hypothesis approaches, our approach avoids
exponential memory complexity and is fast enough for realtime performance.

We show that the proposed method not only allows loop closing errors to be automatically identified, but also that in extreme cases, the “front-end” loop-validation systems can be unnecessary. We demonstrate our system both on standard benchmarks and on the real-world datasets that motivated this work.

Link:
Robotics: Science and Systems(RSS), 2012
LocalLink
http://april.eecs.umich.edu/papers/details.php?name=olson2012rss
http://www.roboticsproceedings.org/rss08/p40.pdf

Lab meeting Dec. 12, 2012 (Alan): A Simple Prior-free Method for Non-Rigid Structure-from-Motion Factorization (CVPR 2012 Best Paper Award)

Title: A Simple Prior-free Method for Non-Rigid Structure-from-Motion Factorization (CVPR 2012 Best Paper Award)
Authors: Yuchao Dai, Hongdong Li, Mingyi He

Abstract:
This paper proposes a simple “prior-free” method for solving non-rigid structure-from-motion factorization problems. Other than using the basic low-rank condition, our method does not assume any extra prior knowledge about the nonrigid scene or about the camera motions. Yet, it runs reliably, produces optimal result, and does not suffer from the inherent basis-ambiguity issue which plagued many conventional nonrigid factorization techniques.

Our method is easy to implement, which involves solving no more than an SDP (semi-definite programming) of small and fixed size, a linear Least-Squares or trace-norm minimization. Extensive experiments have demonstrated that it outperforms most of the existing linear methods of nonrigid factorization. This paper offers not only new theoretical insight, but also a practical, everyday solution, to non-rigid structure-from-motion.

Link

Lab meeting Dec 5th 2012 (Jim): Imitation Learning by Coaching

Title: Imitation Learning by Coaching
Authors: He He, Hal Daumé III and Jason Eisner
Neural Information Processing Systems (NIPS), 2012

Abstract:
... we propose to use a coach that demonstrates easy-to-learn actions for the learner and gradually approaches the oracle. ... We apply our algorithm to cost-sensitive dynamic feature selection, a hard decision problem that considers a user-specified accuracy-cost trade-off. ...

Link

NTU CSIE Talk: [2012-11-09] Dr. Koji Yatani, "A Ph.D. – What does it take?"

Title: A Ph.D. – What does it take?

Date: 2012-11-09 2:20pm

Location: R103

Speaker: Dr. Koji Yatani, Microsoft Research Asia

 

Abstract:

 

Getting a Ph.D. surely needs long effort, but why? Of course, research takes time, but a Ph.D. is not just about research. A Ph.D. student needs to be more than just a research person to be a successful Ph.D. This talk is not about a collection of my research projects (although I will introduce some of them a bit); rather, a collection of my experience in research at University of Toronto, Microsoft Research Asia, and industry labs where I did my internship. Through this talk, I will attempt to share my thoughts on what I believe a Ph.D. student should do and learn before getting her Ph.D. Your honest discussions, opinions and feedback would be greatly appreciated.

 

Biography: 

 

Dr. Koji Yatani (http://yatani.jp) is an associate researcher in Human-Computer Interaction Group at Microsoft Research Asia. His main research interests lie in Human-Computer Interaction (HCI) and its intersections with Ubiquitous Computing and Computational Linguistics. More specifically, he is interested in designing new forms of interacting with mobile devices, and developing new hardware and sensing technologies to support user interactions in mobile/ubiquitous computing environments. He is also interested in developing interactive systems and exploring new applications using computational linguistics methods.

 

He received B.Eng. and M.Sci. from University of Tokyo in 2003 and 2005, respectively, and his Ph.D. in Computer Science from University of Toronto in 2011. On November 2011, he joined HCI group at Microsoft Research Asia in Beijing. He was a recipient of NTT Docomo Scholarship (October 2003 -- March 2005), and Japan Society for the Promotion of Science Research Fellowship for Young Scientists (April 2005 -- March 2006). He received the Best Paper Award at CHI 2011. He served as a program committee on CHI 2013, Ubicomp 2012, and WHC 2013. He also served as a Mentoring co-chair on ITS 2012.

Lab meeting Oct 17th 2012 (Hank): Motion Segmentation of Multiple Objects from a Freely Moving Monocular Camera

Lab meeting Oct 17th 2012 (Hank): Motion Segmentation of Multiple Objects from a Freely Moving Monocular Camera

Link

Presented by Hank Lin

From ICRA2012

Authors: Rahul Kumar Namdev, Abhijit Kundu, K Madhava Krishna and C. V. Jawahar


Abstract:
Motion segmentation or segmentation of moving
objects is an inevitable component for mobile robotic systems
such as the case with robots performing SLAM and collision
avoidance in dynamic worlds. This paper proposes an incre-mental motion segmentation system that efficiently segments
multiple moving objects and simultaneously build the map of
the environment using visual SLAM modules. Multiple cues
based on optical flow and two view geometry are integrated
to achieve this segmentation. A dense optical flow algorithm
provides for dense tracking of features. Motion potentials based
on geometry are computed for each of these dense tracks. These
geometric potentials along with optical flow potentials are used
to form a graph like structure. A graph based segmentation
algorithm then clusters together nodes of similar potentials
to form the eventual motion segments. Experimental results
of high quality segmentation on different publicly available
datasets demonstrate the effectiveness of our method.

Lab meeting Oct 3rd 2012 (Gene): Decentralised Cooperative Localisation for Heterogeneous Teams of Mobile Robots

Link

Presented by Chun-Kai (Gene) Chang

From ICRA2011 Australian Centre for Field Robotics, University of Sydney, NSW, Australia

Authors: Tim Bailey, Mitch Bryson, Hua Mu , John Vial, Lachlan McCalman and Hugh Durrant-Whyte


Abstract:
This paper presents a distributed algorithm for
performing joint localisation of a team of robots. The mobile
robots have heterogeneous sensing capabilities, with some having
high quality inertial and exteroceptive sensing, while others have
only low quality sensing or none at all. By sharing information,
a combined estimate of all robot poses is obtained. Interrobot
range-bearing measurements provide the mechanism for
transferring pose information from well-localised vehicles to those
less capable.
In our proposed formulation, high frequency egocentric data
(e.g., odometry, IMU, GPS) is fused locally on each platform. This
is the distributed part of the algorithm. Inter-robot measurements,
and accompanying state estimates, are communicated to a central
server, which generates an optimal minimum mean-squared
estimate of all robot poses. This server is easily duplicated for
full redundant decentralisation. Communication and computation
are efficient due to the sparseness properties of the informationform
Gaussian representation. A team of three indoor mobile
robots equipped with lasers, odometry and inertial sensing provides
experimental verification of the algorithms effectiveness in
combining location information.

Lab meeting Sep 12th 2012 (Chih-Chung): Global motion planning under uncertain motion, sensing,and environment map

[LINK]

Presented by Chih-Chung

From Autonomous Robots, Volume 33, No.3, 2012, pp. 255-272

Authors:
Hanna Kurniawati · Tirthankar Bandyopadhyay ·
Nicholas M. Patrikalakis

Abstract:
Uncertainty in motion planning is often caused by
three main sources: motion error, sensing error, and imperfect
environment map. Despite the significant effect of all
three sources of uncertainty to motion planning problems,
most planners take into account only one or at most two
of them. We propose a new motion planner, called Guided
Cluster Sampling (GCS), that takes into account all three
sources of uncertainty for robots with active sensing capabilities.
GCS uses the Partially Observable Markov Decision
Process (POMDP) framework and the point-based
POMDP approach. Although point-based POMDPs have
shown impressive progress over the past few years, it performs
poorly when the environment map is imperfect. This
poor performance is due to the extremely high dimensional
state space, which translates to the extremely large belief
space B.
We alleviate this problem by constructing a more suitable
sampling distribution based on the observations that when the
robot has active sensing capability, B can be partitioned
into a collection of much smaller sub-spaces, and an optimal
policy can often be generated by sufficient sampling of
a small subset of the collection. Utilizing these observations,
GCS samples B in two-stages, a subspace is sampled from
the collection and then a belief is sampled from the subspace.

It uses information from the set of sampled sub-spaces
and sampled beliefs to guide subsequent sampling. Simulation
results on marine robotics scenarios suggest that GCS
can generate reasonable policies for motion planning problems
with uncertain motion, sensing, and environment map,
that are unsolvable by the best point-based POMDPs today.
Furthermore, GCS handles POMDPs with continuous state,
action, and observation spaces. We show that for a class of
POMDPs that often occur in robot motion planning, given
enough time, GCS converges to the optimal policy.

To the best of our knowledge, this is the first convergence
result for point-based POMDPs with continuous action
space.

Lab Meeting June 6th, 2012 (Chiao-Hui ): Robot Musical Accompaniment: Integrating Audio and Visual Cues for Real-time Synchronization with a Human Flutist

Chiao-Hui will present the following paper:
Robot Musical Accompaniment: Integrating Audio and Visual Cues for Real-time Synchronization with a Human Flutist

Authors: Angelica Lim, Takeshi Mizumoto, Louis-Kenzo Cahier, Takuma Otsuka, Toru Takahashi, Kazunori Komatani, Tetsuya Ogata and Hiroshi G. Okuno

From: IROS 2010

Abstract:
Musicians often have the following problem: they have a music score that requires 2 or more players, but they have no one with whom to practice. So far, score-playing music robots exist, but they lack adaptive abilities to synchronize with fellow players’ tempo variations. In other words, if the human speeds up their play, the robot should also increase its speed. However, computer accompaniment systems allow exactly this kind of adaptive ability. We present a first step towards giving these accompaniment abilities to a music robot. We introduce a new paradigm of beat tracking using 2 types of sensory input – visual and audio – using our own visual cue recognition system and state-of-the-art acoustic onset detection techniques. Preliminary experiments suggest that by coupling these two modalities, a robot accompanist can start and stop a performance in synchrony with a flutist, and detect tempo changes within half a second.

Link

Lab Meeting May 30th, 2012 (Wei-Shin): Progress Report

I will present my current progress on chair manipulation at lab meeting.  (Borrowed account for posting.)

Lab Meeting May 22th, 2012 (Mark):Strong supervision from weak annotation: Interactive training of deformable part models

We propose a framework for large scale learning and annotation of structured models. The system interleaves interactive labeling (where the current model is used to semi-automate the labeling of a new example) and online learning (where a newly labeled example is used to update the current model parameters). This framework is scalable to large datasets and complex image models and is shown to have excellent theoretical and practical properties in terms of train time, optimality guarantees, and bounds on the amount of annotation effort per image. We apply this framework to part-based detection, and introduce a novel algorithm for interactive labeling of deformable part models. The labeling tool updates and displays in real-time the maximum likelihood location of all parts as the user clicks and drags the location of one or more parts. We demonstrate that the system can be used to efficiently and robustly train part and pose detectors on the CUB Birds-200-a challenging dataset of birds in unconstrained pose and environment.
paper link

[Robot Perception and Learning] Meeting 2012/05/02 (Andi):Energy Based Multiple Model Fitting for Non-Rigid Structure from Motion

Energy Based Multiple Model Fitting for Non-Rigid Structure from Motion

Authors: Chris Russell, Joao Fayad, Lourdes Agapito

From: CVPR '11

Abstract: 

In this paper we reformulate the 3D reconstruction of deformable surfaces from monocular video sequences as a labeling problem. We solve simultaneously for the assignment of feature points to multiple local deformation models and the fitting of models to points to minimize a geometric cost, subject to a spatial constraint that neighboring points should also belong to the same model.

Piecewise reconstruction methods rely on features shared between models to enforce global consistency on the 3D surface. To account for this overlap between regions, we consider a super-set of the classic labeling problem in which a set of labels, instead of a single one, is assigned to each variable. We propose a mathematical formulation of this new model and show how it can be efficiently optimized with a variant of -expansion. We demonstrate how this framework can be applied to Non-Rigid Structure from Motion and leads to simpler explanations of the same data. Compared to existing methods run on the same data, our approach has up to half the reconstruction error, and is more robust to over-fitting and outliers.

Link: get paper here

Lab Meeting April 25th, 2012 (David): Progress Report

    I will present my research progress on laser scanner and stereo camera sensor fusion, with application on pedestrian detection.

Lab Meeting April 11th, 2012 (Jeff): Progress Report

The progress of my current work on Failure Tolerance SLAM with RFID tags would be shared during the meeting.

Lab Meeting, March 28, 2012 (Alan): Realtime Multibody Visual SLAM with a Smoothly Moving Monocular Camera (ICCV 2011)

Title: Realtime Multibody Visual SLAM with a Smoothly Moving Monocular Camera

In: 2011 IEEE International Conference on Computer Vision (ICCV 2011)

Authors: Abhijit Kundu, K Madhava Krishna and C. V. Jawahar

Abstract:

This paper presents a realtime, incremental multibody visual SLAM system that allows choosing between full 3D reconstruction or simply tracking of the moving objects. Motion reconstruction of dynamic points or objects from a monocular camera is considered very hard due to well known problems of observability. We attempt to solve the problem with a Bearing only Tracking (BOT) and by integrating multiple cues to avoid observability issues. The BOT is accomplished through a particle filter, and by integrating multiple cues from the reconstruction pipeline. With the help of these cues, many real world scenarios which are considered unobservable with a monocular camera is solved to reasonable accuracy. This enables building of a unified dynamic 3D map of scenes involving multiple moving objects. Tracking and reconstruction is preceded by motion segmentation and detection which makes use of efficient geometric constraints to avoid difficult degenerate motions, where objects move in the epipolar plane. Results reported on multiple challenging real world image sequences verify the efficacy of the proposed framework.

Link: local copy

Lab Meeting Mar. 21, 2012 (Wang Li): Object interaction detection using hand posture cues in an office setting (IJHCS 2011)

Object interaction detection using hand posture cues in an office setting

Brandon Paulson
Danielle Cummings
Tracy Hammond

Abstract

The goal of this paper is to determine if hand posture can be used as a cue to determine the types of interactions a user has with objects in a desk/office environment. Our experiments indicate that (a) hand posture can be used to determine object interaction, with accuracy rates around 97%, and (b) hand posture is dependent upon the individual user when users are allowed to interact with objects as they would naturally.

Paper Link

Lab Meeting March 07, 2012 (Jimmy): Hand-Grip and Body-Loss Impact on RSS Measurements for Localization of Mass Market Devices

Title: Hand-Grip and Body-Loss Impact on RSS Measurements for Localization of Mass Market Devices
Authors: Rosa, F.D.; Li Xu; Nurmi, J.; Pelosi, M.; Laoudias, C.; Terrezza, A.
In: IEEE International Conference on Localization and GNSS (ICL-GNSS), 2011

Abstract
In this paper we present the effect of the hand-grip and the presence of the human body on received signal strength measurements when performing positioning of mass market devices in indoor environments. We demonstrate that the mitigation of both human body and hand-grip influence can enhance the positioning accuracy and that the human factor cannot be neglected in experimental activities with real mobile devices.

[link]

Lab Meeting Feb. 29 (Hank): Creating Household Environment Map for Environment Manipulation Using Color Range Sensors on Environment and Robot

Authors: Yohei Kakiuchi and Ryohei Ueda and Kei Okada and Masayuki Inaba

Abstract— A humanoid robot working in a household environment with people needs to localize and continuously update the locations of obstacles and manipulable objects. Achieving such system, requires strong perception method to efficiently update the frequently changing environment.

We propose a method for mapping a household environment using multiple stereo and depth cameras located on the humanoid head and the environment. The method relies on colored 3D point cloud data computed from the sensors. We achieve robot localization by matching the point clouds from the robot sensor data directly with the environment sensor data. Object detection is performed using Iterative Closest Point (ICP) with a database of known point cloud models. In order to guarantee accurate object detection results, objects are only detected within the robot sensor data. Furthermore, we utilize the environment sensor data to map out of the obstacles as bounding convex hulls.

We show experimental results creating a household environment map with known object labels and estimate the robot position in this map.


[link]

Lab meeting Feb 22(Chih Chung): Motion planning in urban environments (Journal of Field Robotics 2008)

Author: Dave Ferguson, Thomas M. Howard and Maxim Likhachev

Abstract
We present the motion planning framework for an autonomous vehicle navigating through urban environments. Such environments present a number of motion planning challenges, including ultrareliability, high-speed operation, complex intervehicle interaction, parking in large unstructured lots, and constrained maneuvers. Our approach combines a model-predictive trajectory generation algorithm for computing dynamically feasible actions with two higher level planners for generating long-range plans in both on-road and unstructured areas of the environment. In the first part of this article, we describe the underlying trajectory generator and the on-road planning component of this system. We then describe the unstructured planning component of this system used for navigating through parking lots and recovering from anomalous on-road scenarios. Throughout, we provide examples and results from “Boss” an autonomous sport utility vehicle that has driven itself over 3,000 km and competed in, and won, the DARPA Urban Challenge.

[LINK]

Lab Meeting Dec. 29, 2011 (David): Semantic fusion of laser and vision in pedestrian detection (PR 2010)

Lab Meeting Dec. 29, 2011 (David): Semantic fusion of laser and vision in pedestrian detection (PR 2010)

Luciano Oliveira, Urbano Nunes, Paulo Peixoto, Marco Silva, Fernando Moita

Abstract
    Fusion of laser and vision in object detection has been accomplished by two main approaches: (1) independent integration of sensor-driven features or sensor-driven classifiers, or (2) a region of interest (ROI) is found by laser segmentation and an image classifier is used to name the projected ROI. Here, we propose a novel fusion approach based on semantic information, and embodied on many levels. Sensor fusion is based on spatial relationship of parts-based classifiers, being performed via a Markov logic network. The proposed system deals with partial segments, it is able to recover depth information even if the laser fails, and the integration is modeled through contextual information—characteristics not found on previous approaches. Experiments in pedestrian detection demonstrate the effectiveness of our method over data sets gathered in urban scenarios.

Paper Link

Local Link

Lab Meeting Dec. 22, 2011 (Wang Li): Fast Point Feature Histograms (FPFH) for 3D Registration (ICRA 2009)

Fast Point Feature Histograms (FPFH) for 3D Registration

Radu Bogdan Rusu
Nico Blodow
Michael Beetz

Abstract

In this paper, we modify the mathematical expressions of Point Feature Histograms (PFH), and perform a rigorous analysis on their robustness and complexity for the problem of 3D registration. More concretely, we present optimizations that reduce the computation times drastically by either caching previously computed values or by revising their theoretical formulations. The latter results in a new type of local features, called Fast Point Feature Histograms (FPFH), which retain most of the discriminative power of the PFH. Moreover, we propose an algorithm for the online computation of FPFH features, demonstrate their efficiency for 3D registration, and propose a new sample consensus based method for bringing two datasets into the convergence basin of a local non-linear optimizer: SAC-IA (SAmple Consensus Initial Alignment).

Paper Link

Lab Meeting December 22nd, 2011 (Jeff): Towards Semantic SLAM using a Monocular Camera

Title: Towards Semantic SLAM using a Monocular Camera

Authors: Javier Civera, Dorian G´alvez-L´opez, L. Riazuelo, Juan D. Tard´os, and J. M. M. Montiel

Abstract:

Monocular SLAM systems have been mainly focused on producing geometric maps just composed of points or edges; but without any associated meaning or semantic content.
In this paper, we propose a semantic SLAM algorithm that merges in the estimated map traditional meaningless points with known objects. The non-annotated map is built using only the information extracted from a monocular image sequence. The known object models are automatically computed from a sparse set of images gathered by cameras that may be different from the SLAM camera. The models include both visual appearance and tridimensional information. The semantic or annotated part of the map –the objects– are estimated using the information in the image sequence and the precomputed object models.

The proposed algorithm runs an EKF monocular SLAM parallel to an object recognition thread. This latest one informs of the presence of an object in the sequence by searching
for SURF correspondences and checking afterwards their geometric compatibility. When an object is recognized it is inserted in the SLAM map, being its position measured and hence refined by the SLAM algorithm in subsequent frames. Experimental results show real-time performance for a handheld camera imaging a desktop environment and for a camera
mounted in a robot moving in a room-sized scenario.

Link:
IEEE International Conference on Intelligent Robots and Systems(IROS), 2011
LocalLink
http://webdiis.unizar.es/~jcivera/papers/civera_etal_iros11.pdf

Lab Meeting Dec. 15, 2011 (Alan): Two-View Motion Segmentation with Model Selection and Outlier Removal by RANSAC-Enhanced Dirichlet ... (IJCV 2010)

Title: Two-View Motion Segmentation with Model Selection and Outlier Removal by RANSAC-Enhanced Dirichlet Process Mixture Models (IJCV 2010)

Authors: Yong-Dian Jian, Chu-Song Chen

Abstract:
We propose a novel motion segmentation algorithm based on mixture of Dirichlet process (MDP) models. In contrast to previous approaches, we consider motion segmentation and its model selection regarding to the number of motion models as an inseparable problem. Our algorithm can simultaneously infer the number of motion models, estimate the cluster memberships of correspondences, and identify the outliers. The main idea is to use MDP models to fully exploit the geometric consistencies before making premature decisions about the number of motion models. To handle outliers, we incorporate RANSAC into the inference process of MDP models. In the experiments, we compare the proposed algorithm with naive RANSAC, GPCA and Schindler’s method on both synthetic data and real image data. The experimental results show that we can handlemore motions and have satisfactory performance in the presence of various levels of noise and outlier.

Link

Lab Meeting Dec. 8, 2011 (Jim): Execution of a Dual-Object (Pushing) Action with Semantic Event Chains

Title: “Execution of a Dual-Object (Pushing) Action with Semantic Event Chains”
Authors: Aksoy Eren Erdal, Dellen Babette, Tamosiunaite Minija, and Wörgötter Florentin
In IEEE-RAS Int. Conf. on Humanoid Robots, pp.576-583

Abstract:
Here we present a framework for manipulation execution based on the so called “Semantic Event Chain” which is an abstract description of relations between the objects in the scene. It captures the change of those relations during a manipulation and thereby provides the decisive temporal anchor points by which a manipulation is critically defined. Using semantic event chains a model of a manipulation can be learned. We will show that it is possible to add the required control parameters (the spatial anchor points) to this model, which can then be executed by a robot in a fully autonomous way. The process of learning and execution of semantic event chains is explained using a box pushing example

Link

Lab Meeting November 24, 2011 (Hank): A Large-Scale Hierarchical Multi-View RGB-D Object Dataset (ICRA 2011)

Authors: K. Lai, L. Bo, X. Ren, and D. Fox.
Title:A Large-Scale Hierarchical Multi-View RGB-D Object Dataset
In: Proc. of International Conference on Robotics and Automation (ICRA), 2011

Abstract:
Over the last decade, the availability of public image repositories and recognition benchmarks has enabled rapid progress in visual object category and instance detection. Today we are witnessing the birth of a new generation of sensing technologies capable of providing high quality synchronized videos of both color and depth, the RGB-D (Kinectstyle) camera. With its advanced sensing capabilities and the potential for mass adoption, this technology represents an opportunity to dramatically increase robotic object recognition, manipulation, navigation, and interaction capabilities. In this paper, we introduce a large-scale, hierarchical multi-view object dataset collected using an RGB-D camera. The dataset contains 300 objects organized into 51 categories and has been made publicly available to the research community so as to enable rapid progress based on this promising technology. This paper describes the dataset collection procedure and introduces techniques for RGB-D based object recognition and detection, demonstrating that combining color and depth information substantially improves quality of results.

link

Lab Meeting November 24, 2011 (Jimmy): Tracking Mobile Users in Wireless Networks via Semi-Supervised Co-Localization (TPAMI 2011)

Title: Tracking Mobile Users in Wireless Networks via Semi-Supervised Co-Localization
Authors: Jeffrey Junfeng Pan, Sinno Jialin Pan, Jie Yin, Lionel M. Ni, and Qiang Yang
In: TPAMI 2011

Abstract
Recent years have witnessed growing popularity of sensor and sensor-network technologies, supporting important practical applications. One of the fundamental issues is how to accurately locate a user with few labelled data in a wireless sensor network, where a major difficulty arises from the need to label large quantities of user location data, which in turn requires knowledge about the locations of signal transmitters, or access points. To solve this problem, we have developed a novel machine-learning-based approach that combines collaborative filtering with graph-based semi-supervised learning to learn both mobile-users’ locations and the locations of access points. Our framework exploits both labelled and unlabelled data from mobile devices and access points. In our two-phase solution, we first build a manifold-based model from a batch of labelled and unlabelled data in an offline training phase and then use a weighted k-nearest-neighbor method to localize a mobile client in an online localization phase. We extend the two-phase co-localization to an online and incremental model that can deal with labelled and unlabelled data that come sequentially and adapt to environmental changes. Finally, we embed an action model to the framework such that additional kinds of sensor signals can be utilized to further boost the performance of mobile tracking. Compared to other state-of-the-art systems, our framework has been shown to be more accurate while requiring less calibration effort in our experiments performed at three different test-beds.

[pdf]

Lab Meeting November 17, 2011 (Chih-Chung): Motion Planning under Uncertainty for Robotic Tasks with Long Time Horizons (IJRR 2011)

Authors: Hanna Kurniawati, Yanzhu Du, David Hsu and Wee Sun Lee.

Abstract:
Motion planning with imperfect state information is a crucial capability for autonomous robots to operate reliably in uncertain and dynamic environments. Partially observable Markov decision processes (POMDPs) provide a principled general framework for planning under uncertainty. Using probabilistic sampling, point-based POMDP solvers have drastically improved the speed of POMDP planning, enabling us to handle moderately complex robotic tasks. However, robot motion planning tasks with long time horizons remains a severe obstacle for even the fastest point-based POMDP solvers today. This paper proposes Milestone Guided Sampling (MiGS), a new point-based POMDP solver,which exploits state space information to reduce e ective planning horizons. MiGS samples a set of points, called milestones, from a robot's state space and constructs a simpli ed representation of the state space from the sampled milestones. It then uses this representation of the state space to guide
sampling in the belief space and tries to capture the essential features of the belief space with a small number of sampled points. Preliminary results are very promising. We tested MiGS in simulation on several di cult POMDPs that model distinct robotic tasks with long time horizons in both 2-D and 3-D environments. These POMDPs are impossible to solve with the fastest point-based solvers today, but MiGS solved them in a few minutes.

Link

Lab Meeting November 03, 2011 (David): Real-Time Multi-Person Tracking with Detector Assisted Structure Propagation (ICCV'11 Workshop)

Lab Meeting November 03, 2011 (David): Real-Time Multi-Person Tracking with Detector Assisted Structure Propagation (ICCV'11 Workshop)

Authors: Dennis Mitzel and Bastian Leibe

Abstract:
    Classical tracking-by-detection approaches require a robust object detector that needs to be executed in each frame. However the detector is typically the most computationally expensive component, especially if more than one object class needs to be detected. In this paper we investigate how the usage of the object detector can be reduced by using stereo range data for following detected objects over time. To this end we propose a hybrid tracking framework consisting of a stereo based ICP (Iterative Closest Point) tracker and a high-level multi-hypothesis tracker. Initiated by a detector response, the ICP tracker follows individual pedestrians over time using just the raw depth information. Its output is then fed into the high-level tracker that is responsible for solving long-term data association and occlusion handling. In addition, we propose to constrain the detector to run only on some small regions of interest (ROIs) that are extracted from a 3D depth based occupancy map of the scene. The ROIs are tracked over time and only newly appearing ROIs are evaluated by the detector. We present experiments on real stereo sequences recorded from a moving camera setup in urban scenarios and show that our proposed approach achieves state of the art performance

Link

Lab Meeting October 27, 2011 (ShaoChen): A multiple hypothesis people tracker for teams of mobile robots (ICRA 2010)

Title: A multiple hypothesis people tracker for teams of mobile robots (ICRA 2010)

Authors: Tsokas, N.A. and Kyriakopoulos, K.J.

Abstract: This paper tackles the problem of tracking walking people with multiple moving robots equipped with laser rangefinders. We present an adaptation to the classic Multiple Hypothesis Tracking method, which allows for one-to-many associations between targets and measurements in each cycle and is thus capable of operating in a multi-sensor scenario. In the context of two experiments, the successful integration of our tracking algorithm to a dual-robot setup is assessed.

Link

Lab Meeting October 13, 2011 (Alan): A Model-Selection Framework for Multibody Structure-and-Motion of Image Sequences (IJCV 2008)

Title: A Model-Selection Framework for Multibody Structure-and-Motion of Image Sequences (IJCV 2008)

Authors: Konrad Schindler, David Suter and Hanzi Wang

Abstract: Given an image sequence of a scene consisting of multiple rigidly moving objects, multi-body structureand-motion (MSaM) is the task to segment the image feature tracks into the different rigid objects and compute the multiple-view geometry of each object.We present a framework for multibody structure-and-motion based on model selection. In a recover-and-select procedure, a redundant set of hypothetical scene motions is generated. Each subset of this pool of motion candidates is regarded as a possible explanation of the image feature tracks, and the most likely explanation is selected with model selection. The framework is
generic and can be used with any parametric camera model, or with a combination of different models. It can deal with sets of correspondences, which change over time, and it is robust to realistic amounts of outliers. The framework is demonstrated for different camera and scene models.

Link

Lab Meeting October 13th, 2011 (Jeff): Object Mapping, Recognition, and Localization from Tactile Geometry

Title: Object Mapping, Recognition, and Localization from Tactile Geometry

Authors: Zachary Pezzementi, Caitlin Reyda, and Gregory D. Hager

Abstract:

We present a method for performing object recognition using multiple images acquired from a tactile sensor. The method relies on using the tactile sensor as an imaging device, and builds an object representation based on mosaics of tactile measurements. We then describe an algorithm that is able to recognize an object using a small number of tactile sensor readings. Our approach makes extensive use of sequential state estimation techniques from the mobile robotics literature, whereby we view the object recognition problem as one of estimating a consistent location within a set of object maps. We examine and test approaches based on both traditional
particle filtering and histogram filtering. We demonstrate both the mapping and recognition / localization techniques on a set of raised letter shapes using real tactile sensor data.

Link:
IEEE International Conference on Robotics and Automation(ICRA), 2011
LocalLink
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5980363

Lab Meeting Sep. 29, 2011 (Wang Li): A Coarse-to-fine Approach for Fast Deformable Object Detection (CVPR 2011)

A Coarse-to-fine Approach for Fast Deformable Object Detection

Marco Pedersoli
Andrea Vedaldi
Jordi González

Abstract

We present a method that can dramatically accelerate object detection with part based models. The method is based on the observation that the cost of detection is likely to be dominated by the cost of matching each part to the image, and not by the cost of computing the optimal configuration of the parts as commonly assumed. Therefore accelerating detection requires minimizing the number of part-to-image comparisons. To this end we propose a multiple-resolutions hierarchical part based model and a corresponding coarse-to-fine inference procedure that recursively eliminates from the search space unpromising part placements. We evaluate our method extensively on the PASCAL VOC and INRIA datasets, demonstrating a very high increase in the detection speed with little degradation of the accuracy.

Paper Link

Lab Meeting September 22nd, 2011 (Jimmy): Vector Field SLAM

Title: Vector Field SLAM
Authors: Jens-Steffen Gutmann, Gabriel Brisson, Ethan Eade, Philip Fong and Mario Munich
In: ICRA 2010

Abstract
Localization in unknown environments using low-cost sensors remains a challenge. This paper presents a new localization approach that learns the spatial variation of an observed continuous signal. We model the signal as a piecewise linear function and estimate its parameters using a simultaneous localization and mapping (SLAM) approach. We apply our framework to a sensor measuring bearing to active beacons where measurements are systematically distorted due to occlusion and signal reflections of walls and other objects present in the environment. Experimental results from running GraphSLAM and EKF-SLAM on manually collected sensor measurements as well as on data recorded on a vacuum-cleaner robot validate our model.

[pdf]

Lab Meeting September 22nd, 2011 (Jim): Learning the semantics of object–action relations by observation

Title: Learning the semantics of object–action relations by observation
Author: Eren Erdal Aksoy, Alexey Abramov, Johannes Dörr, Kejun Ning, Babette Dellen, and Florentin Wörgötter
The International Journal of Robotics Research 2011;30 1229-1249

Abstract:
Recognizing manipulations performed by a human and the transfer and execution of this by a robot is a difficult problem. We address this in the current study by introducing a novel representation of the relations between objects at decisive time points during a manipulation. Thereby, we encode the essential changes in a visual scenery in a condensed way such that a robot can recognize and learn a manipulation without prior object knowledge. To achieve this we continuously track image segments in the video and construct a dynamic graph sequence. Topological transitions of those graphs occur whenever a spatial relation between some segments has changed in a discontinuous way and these moments are stored in a transition matrix called the semantic event chain (SEC). We demonstrate that these time points are highly descriptive for distinguishing between different manipulations. Employing simple sub-string search algorithms, SECs can be compared and type-similar manipulations can be recognized with high confidence. As the approach is generic, statistical learning can be used to find the archetypal SEC of a given manipulation class. ...

http://ijr.sagepub.com/content/30/10/1229.full.pdf+html

Lab Meeting September 9th, 2011 (Steven): Learning Generic Invariances in Object Recognition: Translation and Scale

Title: Learning Generic Invariances in Object Recognition: Translation and Scale

Authors: Joel Z Leibo, Jim Mutch, Lorenzo Rosasco, Shimon Ullman4, and Tomaso Poggio

Abstract:

Invariance to various transformations is key to object recognition but existing definitions of invariance are somewhat confusing while discussions of invariance are often confused. In this report, we provide an operational definition of invariance by formally defining perceptual tasks as classification problems. The definition should be appropriate for physiology, psychophysics and computational modeling.

For any specific object, invariance can be trivially “learned” by memorizing a sufficient number of example images of the transformed object. While our formal definition of invariance also covers such cases, this report focuses instead on invariance from very few images and mostly on invariances from one example. Image-plane invariances – such as translation, rotation and scaling – can be computed from a single image for any object. They are called generic since in principle they can be hardwired or learned (during development) for any object.

In this perspective, we characterize the invariance range of a class of feedforward architectures for visual recognition that mimic the hierarchical organization of the ventral stream.

We show that this class of models achieves essentially perfect translation and scaling invariance for novel images. In this architecture a new image is represented in terms of weights of ”templates” (e.g. “centers” or “basis functions”) at each level in the hierarchy. Such a representation inherits the invariance of each template, which is implemented through replication of the corresponding “simple” units across positions or scales and their “association” in a “complex” unit. We show simulations on real images that characterize the type and number of templates needed to support the invariant recognition of novel objects. We find that 1) the templates need not be visually similar to the target objects and that 2) a very small number of them is sufficient for good recognition.
These somewhat surprising empirical results have intriguing implications for the learning of invariant recognition during the development of a biological organism, such as a human baby. In particular, we conjecture that invariance to translation and scale may be learned by the association – through temporal contiguity – of a small number of primal templates, that is patches extracted from the images of an object moving on the retina across positions and scales. The number of templates can later be augmented by bootstrapping mechanisms using the correspondence provided by the primal templates – without the need of temporal contiguity.

Link

Lab Meeting September 9th, 2011 (Chih Chung): Identification and Representation of Homotopy (RSS 2011 Best paper)

title: Identification and Representation of Homotopy
Classes of Trajectories for Search-based Path
Planning in 3D

Authors: Subhrajit Bhattacharya, Maxim Likhachev and Vijay Kumar

Abstract: There are many applications in motion planning
where it is important to consider and distinguish between
different homotopy classes of trajectories. Two trajectories are
homotopic if one trajectory can be continuously deformed into
another without passing through an obstacle, and a homotopy
class is a collection of homotopic trajectories. In this paper
we consider the problem of robot exploration and planning in
three-dimensional configuration spaces to (a) identify and classify
different homotopy classes; and (b) plan trajectories constrained
to certain homotopy classes or avoiding specified homotopy
classes. In previous work [1] we have solved this problem for
two-dimensional, static environments using the Cauchy Integral
Theorem in concert with graph search techniques. The robot
workspace is mapped to the complex plane and obstacles are poles
in this plane. The Residue Theorem allows the use of integration
along the path to distinguish between trajectories in different
homotopy classes. However, this idea is fundamentally limited
to two dimensions. In this work we develop new techniques to
solve the same problem, but in three dimensions, using theorems
from electromagnetism. The Biot-Savart law lets us design an
appropriate vector field, the line integral of which, using the
integral form of Ampere’s Law, encodes information about
homotopy classes in three dimensions. Skeletons of obstacles
in the robot world are extracted and are modeled by currentcarrying
conductors. We describe the development of a practical
graph-search based planning tool with theoretical guarantees
by combining integration theory with search techniques, and
illustrate it with examples in three-dimensional spaces such as
two-dimensional, dynamic environments and three-dimensional
static environments.


link

Lab Meeting September 2nd, 2011 (David): Multiclass Multimodal Detection and Tracking in Urban Environments

Title: Multiclass Multimodal Detection and Tracking in Urban Environments

Author: Luciano Spinello, Rudolph Triebel and Roland Siegwart

Abstract:
This paper presents a novel approach to detect and track people and cars based on the combined information retrieved from a camera and a laser range scanner. Laser data points are classified by using boosted Conditional Random Fields (CRF), while the image based detector uses an extension of the Implicit Shape Model (ISM), which learns a codebook of local descriptors from a set of hand-labeled images and uses them to vote for centers of detected objects. Our extensions to ISM include the learning of object parts and template masks to obtain more distinctive votes for the particular object classes. The detections from both sensors are then fused and the objects are tracked using a Kalman Filter with multiple motion models. Experiments conducted in real-world urban scenarios demonstrate the effectiveness of our approach.

Link:
IJRR copy
localcopy

Lab Meeting August 19th, 2011 (Jeff): A Robust Qualitative Planner for Mobile Robot Navigation Using Human-Provided Maps

Title: A Robust Qualitative Planner for Mobile Robot Navigation Using Human-Provided Maps

Authors: Danelle C. Shah and Mark E. Campbell

Abstract:

A novel method for controlling a mobile robot using qualitative inputs in the context of an approximate map, such as one sketched by a human, is presented. By defining a desired trajectory with respect to observable landmarks, human operators can send semi-autonomous robots into areas for which a truth map is not available. Waypoint planning is formulated as a quadratic optimization problem, resulting in robot trajectories in the true environment that are qualitatively similar to those provided by the human. The algorithm is implemented both in simulation and on a mobile robot platform in several different environments. A sensitivity analysis is performed, illustrating how the method is robust to uncertainties, even large sketch distortions, and allows the robot to adapt and re-plan according to its most current perception of the world.

Link:
IEEE International Conference on Robotics and Automation(ICRA), 2011
LocalLink

A robot that flies like a bird | Video on TED.com

A robot that flies like a bird | Video on TED.com

Lab meeting June 29th (Jim): A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

Title: A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

Stephane Ross, Geoffrey Gordon, and J. Andrew (Drew) Bagnell

Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), April, 2011.

Abstracts:
Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. ... In this paper, we propose a new iterative algorithm, which trains a stationary deterministic policy, that can be seen as a no regret algorithm in an online learning setting. We show that any such no regret algorithm, combined with additional reduction assumptions, must find a policy with good performance under the distribution of observations it induces in such sequential settings.

Link

Lab Meeting June 22th (Chih-Chung):Minimum Snap Trajectory Generation and Control for Quadrotors (ICRA2011,best paper)

Title: Minimum Snap Trajectory Generation and Control for Quadrotors

Authors: Daniel Mellinger and Vijay Kumar

Abstracts:
We address the controller design and the trajectory
generation for a quadrotor maneuvering in three
dimensions in a tightly constrained setting typical of indoor
environments. In such settings, it is necessary to allow for
significant excursions of the attitude from the hover state and
small angle approximations cannot be justified for the roll
and pitch. We develop an algorithm that enables the real-time
generation of optimal trajectories through a sequence of 3-D
positions and yaw angles, while ensuring safe passage through
specified corridors and satisfying constraints on velocities,
accelerations and inputs. A nonlinear controller ensures the
faithful tracking of these trajectories. Experimental results
illustrate the application of the method to fast motion (5-10
body lengths/second) in three-dimensional slalom courses.

[link]

Lab Meeting June 15th (Shao-Chen): Distributed Robust Data Fusion Based on Dynamic Voting (ICRA2011)

Title: Distributed Robust Data Fusion Based on Dynamic Voting

Authors: Eduardo Montijano, Sonia Mart´ınez and Carlos Sagues

Abstract:

Data association mistakes, estimation and measurement errors are some of the factors that can contribute to incorrect observations in robotic sensor networks. In order to act reliably, a robotic network must be able to fuse and correct its perception of the world by discarding any outlier information. This is a difficult task if the network is to be deployed remotely and the robots do not have access to groundtruth sites or manual calibration. In this paper, we present a novel, distributed scheme for robust data fusion in autonomous robotic networks. The proposed method adapts the RANSAC algorithm to exploit measurement redundancy, and enables robots determine an inlier observation with local communications. Different hypotheses are generated and voted for using a dynamic consensus algorithm. As the hypotheses are computed, the robots can change their opinion making the voting process dynamic. Assuming that at least one hypothesis is initialized with only inliers, we show that the method converges to the maximum likelihood of all the inlier observations in a general instance. Several simulations exhibit the good performance of the algorithm, which also gives acceptable results in situations where the conditions to guarantee convergence do not hold.

[link]

Lab Meeting June 15th (David): Sparse Scene Flow Segmentation for Moving Object Detection (Intelligent Vehicles Symposium 2011)

Title: Sparse Scene Flow Segmentation for Moving Object Detection (Intelligent Vehicles Symposium 2011)

Authors: P. Lenz, J. Ziegler, A. Geiger, M. Roser

Abstract:
Modern driver assistance systems such as collision avoidance or intersection assistance need reliable information on the current environment. Extracting such information from camera-based systems is a complex and challenging task for inner city taffic scenarios. This paper presents an approach for object detection utilizing sparse scene flow. For consecutive stereo images taken from a moving vehicle, corresponding interest points are extracted. Thus, for every interest point, disparity and optical flow values are known and consequently, scene flow can be calculated. Adjacent interest points describing a similar scene flow are considered to belong to one rigid object. The proposed method does not rely on object classes and allows for a robust detection of dynamic objects in traffic scenes. Leading vehicles are continuously detected for several frames. Oncoming objects are detected within five frames after their appearance.

Link: http://www.rainsoft.de/publications/iv11b.pdf

Lab Meeting June 8th, 2011 (Jeff): Incremental Construction of the Saturated-GVG for Multi-Hypothesis Topological SLAM

Title: Incremental Construction of the Saturated-GVG for Multi-Hypothesis
Topological SLAM

Authors: Tong Tao, Stephen Tully, George Kantor, and Howie Choset

Abstract:

The generalized Voronoi graph (GVG) is a topological representation of an environment that can be incrementally constructed with a mobile robot using sensor-based control. However, because of sensor range limitations, the GVG control law will fail when the robot moves into a large open area. This paper discusses an extended GVG approach to topological navigation and mapping: the saturated generalized Voronoi graph (S-GVG), for which the robot employs an additional wall-following behavior to navigate along obstacles at the range limit of the sensor. In this paper, we build upon previous work related to the S-GVG and provide two important contributions: 1) a rigorous discussion of the control laws and algorithm modifications that are necessary for incremental construction of the S-GVG with a mobile robot, and 2) a method for incorporating the S-GVG into a novel multi-hypothesis SLAM algorithm for loop-closing and localization. Experiments with a wheeled mobile robot in an office-like environment validate the ffectiveness of the proposed approach.

Link:
IEEE International Conference on Robotics and Automation(ICRA), 2011
http://www.cs.cmu.edu/~biorobotics/papers/icra11_tao.pdf
LocalLink

Lab Meeting June 1, 2011 (Alan): Semantic Structure from Motion (CVPR 2011)

Title: Semantic Structure from Motion (CVPR 2011)

Authors: Sid Yingze Bao and Silvio Savarese

Abstract

Conventional rigid structure from motion (SFM) addresses the problem of recovering the camera parameters (motion) and the 3D locations (structure) of scene points, given observed 2D image feature points. In this paper, we propose a new formulation called Semantic Structure From Motion (SSFM). In addition to the geometrical constraints provided by SFM, SSFM takes advantage of both semantic and geometrical properties associated with objects in the scene (Fig. 1). These properties allow us to recover not only the structure and motion but also the 3D locations, poses, and categories of objects in the scene. We cast this problem as a max-likelihood problem where geometry (cameras, points, objects) and semantic information (object classes) are simultaneously estimated. The key intuition is that, in addition to image features, the measurements of objects across views provide additional geometrical constraints that relate cameras and scene parameters. These constraints make the geometry estimation process more robust and, in turn, make object detection more accurate. Our framework has the unique ability to: i) estimate camera poses only from object detections, ii) enhance camera pose estimation, compared to feature-point-based SFM algorithms, iii) improve object detections given multiple uncalibrated images, compared to independently detecting objects in single images. Extensive quantitative results on three datasets – LiDAR cars, street-view pedestrians, and Kinect office desktop – verify our theoretical claims.

Link