NTU CSIE Talk: [2012-11-09] Dr. Koji Yatani, "A Ph.D. – What does it take?"

Title: A Ph.D. – What does it take?

Date: 2012-11-09 2:20pm

Location: R103

Speaker: Dr. Koji Yatani, Microsoft Research Asia

 

Abstract:

 

Getting a Ph.D. surely needs long effort, but why? Of course, research takes time, but a Ph.D. is not just about research. A Ph.D. student needs to be more than just a research person to be a successful Ph.D. This talk is not about a collection of my research projects (although I will introduce some of them a bit); rather, a collection of my experience in research at University of Toronto, Microsoft Research Asia, and industry labs where I did my internship. Through this talk, I will attempt to share my thoughts on what I believe a Ph.D. student should do and learn before getting her Ph.D. Your honest discussions, opinions and feedback would be greatly appreciated.

 

Biography: 

 

Dr. Koji Yatani (http://yatani.jp) is an associate researcher in Human-Computer Interaction Group at Microsoft Research Asia. His main research interests lie in Human-Computer Interaction (HCI) and its intersections with Ubiquitous Computing and Computational Linguistics. More specifically, he is interested in designing new forms of interacting with mobile devices, and developing new hardware and sensing technologies to support user interactions in mobile/ubiquitous computing environments. He is also interested in developing interactive systems and exploring new applications using computational linguistics methods.

 

He received B.Eng. and M.Sci. from University of Tokyo in 2003 and 2005, respectively, and his Ph.D. in Computer Science from University of Toronto in 2011. On November 2011, he joined HCI group at Microsoft Research Asia in Beijing. He was a recipient of NTT Docomo Scholarship (October 2003 -- March 2005), and Japan Society for the Promotion of Science Research Fellowship for Young Scientists (April 2005 -- March 2006). He received the Best Paper Award at CHI 2011. He served as a program committee on CHI 2013, Ubicomp 2012, and WHC 2013. He also served as a Mentoring co-chair on ITS 2012.

Lab meeting Oct 17th 2012 (Hank): Motion Segmentation of Multiple Objects from a Freely Moving Monocular Camera

Lab meeting Oct 17th 2012 (Hank): Motion Segmentation of Multiple Objects from a Freely Moving Monocular Camera

Link

Presented by Hank Lin

From ICRA2012

Authors: Rahul Kumar Namdev, Abhijit Kundu, K Madhava Krishna and C. V. Jawahar


Abstract:
Motion segmentation or segmentation of moving
objects is an inevitable component for mobile robotic systems
such as the case with robots performing SLAM and collision
avoidance in dynamic worlds. This paper proposes an incre-mental motion segmentation system that efficiently segments
multiple moving objects and simultaneously build the map of
the environment using visual SLAM modules. Multiple cues
based on optical flow and two view geometry are integrated
to achieve this segmentation. A dense optical flow algorithm
provides for dense tracking of features. Motion potentials based
on geometry are computed for each of these dense tracks. These
geometric potentials along with optical flow potentials are used
to form a graph like structure. A graph based segmentation
algorithm then clusters together nodes of similar potentials
to form the eventual motion segments. Experimental results
of high quality segmentation on different publicly available
datasets demonstrate the effectiveness of our method.

Lab meeting Oct 3rd 2012 (Gene): Decentralised Cooperative Localisation for Heterogeneous Teams of Mobile Robots

Link

Presented by Chun-Kai (Gene) Chang

From ICRA2011 Australian Centre for Field Robotics, University of Sydney, NSW, Australia

Authors: Tim Bailey, Mitch Bryson, Hua Mu , John Vial, Lachlan McCalman and Hugh Durrant-Whyte


Abstract:
This paper presents a distributed algorithm for
performing joint localisation of a team of robots. The mobile
robots have heterogeneous sensing capabilities, with some having
high quality inertial and exteroceptive sensing, while others have
only low quality sensing or none at all. By sharing information,
a combined estimate of all robot poses is obtained. Interrobot
range-bearing measurements provide the mechanism for
transferring pose information from well-localised vehicles to those
less capable.
In our proposed formulation, high frequency egocentric data
(e.g., odometry, IMU, GPS) is fused locally on each platform. This
is the distributed part of the algorithm. Inter-robot measurements,
and accompanying state estimates, are communicated to a central
server, which generates an optimal minimum mean-squared
estimate of all robot poses. This server is easily duplicated for
full redundant decentralisation. Communication and computation
are efficient due to the sparseness properties of the informationform
Gaussian representation. A team of three indoor mobile
robots equipped with lasers, odometry and inertial sensing provides
experimental verification of the algorithms effectiveness in
combining location information.

Lab meeting Sep 12th 2012 (Chih-Chung): Global motion planning under uncertain motion, sensing,and environment map

[LINK]

Presented by Chih-Chung

From Autonomous Robots, Volume 33, No.3, 2012, pp. 255-272

Authors:
Hanna Kurniawati · Tirthankar Bandyopadhyay ·
Nicholas M. Patrikalakis

Abstract:
Uncertainty in motion planning is often caused by
three main sources: motion error, sensing error, and imperfect
environment map. Despite the significant effect of all
three sources of uncertainty to motion planning problems,
most planners take into account only one or at most two
of them. We propose a new motion planner, called Guided
Cluster Sampling (GCS), that takes into account all three
sources of uncertainty for robots with active sensing capabilities.
GCS uses the Partially Observable Markov Decision
Process (POMDP) framework and the point-based
POMDP approach. Although point-based POMDPs have
shown impressive progress over the past few years, it performs
poorly when the environment map is imperfect. This
poor performance is due to the extremely high dimensional
state space, which translates to the extremely large belief
space B.
We alleviate this problem by constructing a more suitable
sampling distribution based on the observations that when the
robot has active sensing capability, B can be partitioned
into a collection of much smaller sub-spaces, and an optimal
policy can often be generated by sufficient sampling of
a small subset of the collection. Utilizing these observations,
GCS samples B in two-stages, a subspace is sampled from
the collection and then a belief is sampled from the subspace.

It uses information from the set of sampled sub-spaces
and sampled beliefs to guide subsequent sampling. Simulation
results on marine robotics scenarios suggest that GCS
can generate reasonable policies for motion planning problems
with uncertain motion, sensing, and environment map,
that are unsolvable by the best point-based POMDPs today.
Furthermore, GCS handles POMDPs with continuous state,
action, and observation spaces. We show that for a class of
POMDPs that often occur in robot motion planning, given
enough time, GCS converges to the optimal policy.

To the best of our knowledge, this is the first convergence
result for point-based POMDPs with continuous action
space.

Lab Meeting June 6th, 2012 (Chiao-Hui ): Robot Musical Accompaniment: Integrating Audio and Visual Cues for Real-time Synchronization with a Human Flutist

Chiao-Hui will present the following paper:
Robot Musical Accompaniment: Integrating Audio and Visual Cues for Real-time Synchronization with a Human Flutist

Authors: Angelica Lim, Takeshi Mizumoto, Louis-Kenzo Cahier, Takuma Otsuka, Toru Takahashi, Kazunori Komatani, Tetsuya Ogata and Hiroshi G. Okuno

From: IROS 2010

Abstract:
Musicians often have the following problem: they have a music score that requires 2 or more players, but they have no one with whom to practice. So far, score-playing music robots exist, but they lack adaptive abilities to synchronize with fellow players’ tempo variations. In other words, if the human speeds up their play, the robot should also increase its speed. However, computer accompaniment systems allow exactly this kind of adaptive ability. We present a first step towards giving these accompaniment abilities to a music robot. We introduce a new paradigm of beat tracking using 2 types of sensory input – visual and audio – using our own visual cue recognition system and state-of-the-art acoustic onset detection techniques. Preliminary experiments suggest that by coupling these two modalities, a robot accompanist can start and stop a performance in synchrony with a flutist, and detect tempo changes within half a second.

Link

Lab Meeting May 30th, 2012 (Wei-Shin): Progress Report

I will present my current progress on chair manipulation at lab meeting.  (Borrowed account for posting.)

Lab Meeting May 22th, 2012 (Mark):Strong supervision from weak annotation: Interactive training of deformable part models

We propose a framework for large scale learning and annotation of structured models. The system interleaves interactive labeling (where the current model is used to semi-automate the labeling of a new example) and online learning (where a newly labeled example is used to update the current model parameters). This framework is scalable to large datasets and complex image models and is shown to have excellent theoretical and practical properties in terms of train time, optimality guarantees, and bounds on the amount of annotation effort per image. We apply this framework to part-based detection, and introduce a novel algorithm for interactive labeling of deformable part models. The labeling tool updates and displays in real-time the maximum likelihood location of all parts as the user clicks and drags the location of one or more parts. We demonstrate that the system can be used to efficiently and robustly train part and pose detectors on the CUB Birds-200-a challenging dataset of birds in unconstrained pose and environment.
paper link

[Robot Perception and Learning] Meeting 2012/05/02 (Andi):Energy Based Multiple Model Fitting for Non-Rigid Structure from Motion

Energy Based Multiple Model Fitting for Non-Rigid Structure from Motion

Authors: Chris Russell, Joao Fayad, Lourdes Agapito

From: CVPR '11

Abstract: 

In this paper we reformulate the 3D reconstruction of deformable surfaces from monocular video sequences as a labeling problem. We solve simultaneously for the assignment of feature points to multiple local deformation models and the fitting of models to points to minimize a geometric cost, subject to a spatial constraint that neighboring points should also belong to the same model.

Piecewise reconstruction methods rely on features shared between models to enforce global consistency on the 3D surface. To account for this overlap between regions, we consider a super-set of the classic labeling problem in which a set of labels, instead of a single one, is assigned to each variable. We propose a mathematical formulation of this new model and show how it can be efficiently optimized with a variant of -expansion. We demonstrate how this framework can be applied to Non-Rigid Structure from Motion and leads to simpler explanations of the same data. Compared to existing methods run on the same data, our approach has up to half the reconstruction error, and is more robust to over-fitting and outliers.

Link: get paper here

Lab Meeting April 25th, 2012 (David): Progress Report

    I will present my research progress on laser scanner and stereo camera sensor fusion, with application on pedestrian detection.

Lab Meeting April 11th, 2012 (Jeff): Progress Report

The progress of my current work on Failure Tolerance SLAM with RFID tags would be shared during the meeting.

Lab Meeting, March 28, 2012 (Alan): Realtime Multibody Visual SLAM with a Smoothly Moving Monocular Camera (ICCV 2011)

Title: Realtime Multibody Visual SLAM with a Smoothly Moving Monocular Camera

In: 2011 IEEE International Conference on Computer Vision (ICCV 2011)

Authors: Abhijit Kundu, K Madhava Krishna and C. V. Jawahar

Abstract:

This paper presents a realtime, incremental multibody visual SLAM system that allows choosing between full 3D reconstruction or simply tracking of the moving objects. Motion reconstruction of dynamic points or objects from a monocular camera is considered very hard due to well known problems of observability. We attempt to solve the problem with a Bearing only Tracking (BOT) and by integrating multiple cues to avoid observability issues. The BOT is accomplished through a particle filter, and by integrating multiple cues from the reconstruction pipeline. With the help of these cues, many real world scenarios which are considered unobservable with a monocular camera is solved to reasonable accuracy. This enables building of a unified dynamic 3D map of scenes involving multiple moving objects. Tracking and reconstruction is preceded by motion segmentation and detection which makes use of efficient geometric constraints to avoid difficult degenerate motions, where objects move in the epipolar plane. Results reported on multiple challenging real world image sequences verify the efficacy of the proposed framework.

Link: local copy

Lab Meeting Mar. 21, 2012 (Wang Li): Object interaction detection using hand posture cues in an office setting (IJHCS 2011)

Object interaction detection using hand posture cues in an office setting

Brandon Paulson
Danielle Cummings
Tracy Hammond

Abstract

The goal of this paper is to determine if hand posture can be used as a cue to determine the types of interactions a user has with objects in a desk/office environment. Our experiments indicate that (a) hand posture can be used to determine object interaction, with accuracy rates around 97%, and (b) hand posture is dependent upon the individual user when users are allowed to interact with objects as they would naturally.

Paper Link

Lab Meeting March 07, 2012 (Jimmy): Hand-Grip and Body-Loss Impact on RSS Measurements for Localization of Mass Market Devices

Title: Hand-Grip and Body-Loss Impact on RSS Measurements for Localization of Mass Market Devices
Authors: Rosa, F.D.; Li Xu; Nurmi, J.; Pelosi, M.; Laoudias, C.; Terrezza, A.
In: IEEE International Conference on Localization and GNSS (ICL-GNSS), 2011

Abstract
In this paper we present the effect of the hand-grip and the presence of the human body on received signal strength measurements when performing positioning of mass market devices in indoor environments. We demonstrate that the mitigation of both human body and hand-grip influence can enhance the positioning accuracy and that the human factor cannot be neglected in experimental activities with real mobile devices.

[link]

Lab Meeting Feb. 29 (Hank): Creating Household Environment Map for Environment Manipulation Using Color Range Sensors on Environment and Robot

Authors: Yohei Kakiuchi and Ryohei Ueda and Kei Okada and Masayuki Inaba

Abstract— A humanoid robot working in a household environment with people needs to localize and continuously update the locations of obstacles and manipulable objects. Achieving such system, requires strong perception method to efficiently update the frequently changing environment.

We propose a method for mapping a household environment using multiple stereo and depth cameras located on the humanoid head and the environment. The method relies on colored 3D point cloud data computed from the sensors. We achieve robot localization by matching the point clouds from the robot sensor data directly with the environment sensor data. Object detection is performed using Iterative Closest Point (ICP) with a database of known point cloud models. In order to guarantee accurate object detection results, objects are only detected within the robot sensor data. Furthermore, we utilize the environment sensor data to map out of the obstacles as bounding convex hulls.

We show experimental results creating a household environment map with known object labels and estimate the robot position in this map.


[link]

Lab meeting Feb 22(Chih Chung): Motion planning in urban environments (Journal of Field Robotics 2008)

Author: Dave Ferguson, Thomas M. Howard and Maxim Likhachev

Abstract
We present the motion planning framework for an autonomous vehicle navigating through urban environments. Such environments present a number of motion planning challenges, including ultrareliability, high-speed operation, complex intervehicle interaction, parking in large unstructured lots, and constrained maneuvers. Our approach combines a model-predictive trajectory generation algorithm for computing dynamically feasible actions with two higher level planners for generating long-range plans in both on-road and unstructured areas of the environment. In the first part of this article, we describe the underlying trajectory generator and the on-road planning component of this system. We then describe the unstructured planning component of this system used for navigating through parking lots and recovering from anomalous on-road scenarios. Throughout, we provide examples and results from “Boss” an autonomous sport utility vehicle that has driven itself over 3,000 km and competed in, and won, the DARPA Urban Challenge.

[LINK]

Lab Meeting Dec. 29, 2011 (David): Semantic fusion of laser and vision in pedestrian detection (PR 2010)

Lab Meeting Dec. 29, 2011 (David): Semantic fusion of laser and vision in pedestrian detection (PR 2010)

Luciano Oliveira, Urbano Nunes, Paulo Peixoto, Marco Silva, Fernando Moita

Abstract
    Fusion of laser and vision in object detection has been accomplished by two main approaches: (1) independent integration of sensor-driven features or sensor-driven classifiers, or (2) a region of interest (ROI) is found by laser segmentation and an image classifier is used to name the projected ROI. Here, we propose a novel fusion approach based on semantic information, and embodied on many levels. Sensor fusion is based on spatial relationship of parts-based classifiers, being performed via a Markov logic network. The proposed system deals with partial segments, it is able to recover depth information even if the laser fails, and the integration is modeled through contextual information—characteristics not found on previous approaches. Experiments in pedestrian detection demonstrate the effectiveness of our method over data sets gathered in urban scenarios.

Paper Link

Local Link

Lab Meeting Dec. 22, 2011 (Wang Li): Fast Point Feature Histograms (FPFH) for 3D Registration (ICRA 2009)

Fast Point Feature Histograms (FPFH) for 3D Registration

Radu Bogdan Rusu
Nico Blodow
Michael Beetz

Abstract

In this paper, we modify the mathematical expressions of Point Feature Histograms (PFH), and perform a rigorous analysis on their robustness and complexity for the problem of 3D registration. More concretely, we present optimizations that reduce the computation times drastically by either caching previously computed values or by revising their theoretical formulations. The latter results in a new type of local features, called Fast Point Feature Histograms (FPFH), which retain most of the discriminative power of the PFH. Moreover, we propose an algorithm for the online computation of FPFH features, demonstrate their efficiency for 3D registration, and propose a new sample consensus based method for bringing two datasets into the convergence basin of a local non-linear optimizer: SAC-IA (SAmple Consensus Initial Alignment).

Paper Link

Lab Meeting December 22nd, 2011 (Jeff): Towards Semantic SLAM using a Monocular Camera

Title: Towards Semantic SLAM using a Monocular Camera

Authors: Javier Civera, Dorian G´alvez-L´opez, L. Riazuelo, Juan D. Tard´os, and J. M. M. Montiel

Abstract:

Monocular SLAM systems have been mainly focused on producing geometric maps just composed of points or edges; but without any associated meaning or semantic content.
In this paper, we propose a semantic SLAM algorithm that merges in the estimated map traditional meaningless points with known objects. The non-annotated map is built using only the information extracted from a monocular image sequence. The known object models are automatically computed from a sparse set of images gathered by cameras that may be different from the SLAM camera. The models include both visual appearance and tridimensional information. The semantic or annotated part of the map –the objects– are estimated using the information in the image sequence and the precomputed object models.

The proposed algorithm runs an EKF monocular SLAM parallel to an object recognition thread. This latest one informs of the presence of an object in the sequence by searching
for SURF correspondences and checking afterwards their geometric compatibility. When an object is recognized it is inserted in the SLAM map, being its position measured and hence refined by the SLAM algorithm in subsequent frames. Experimental results show real-time performance for a handheld camera imaging a desktop environment and for a camera
mounted in a robot moving in a room-sized scenario.

Link:
IEEE International Conference on Intelligent Robots and Systems(IROS), 2011
LocalLink
http://webdiis.unizar.es/~jcivera/papers/civera_etal_iros11.pdf

Lab Meeting Dec. 15, 2011 (Alan): Two-View Motion Segmentation with Model Selection and Outlier Removal by RANSAC-Enhanced Dirichlet ... (IJCV 2010)

Title: Two-View Motion Segmentation with Model Selection and Outlier Removal by RANSAC-Enhanced Dirichlet Process Mixture Models (IJCV 2010)

Authors: Yong-Dian Jian, Chu-Song Chen

Abstract:
We propose a novel motion segmentation algorithm based on mixture of Dirichlet process (MDP) models. In contrast to previous approaches, we consider motion segmentation and its model selection regarding to the number of motion models as an inseparable problem. Our algorithm can simultaneously infer the number of motion models, estimate the cluster memberships of correspondences, and identify the outliers. The main idea is to use MDP models to fully exploit the geometric consistencies before making premature decisions about the number of motion models. To handle outliers, we incorporate RANSAC into the inference process of MDP models. In the experiments, we compare the proposed algorithm with naive RANSAC, GPCA and Schindler’s method on both synthetic data and real image data. The experimental results show that we can handlemore motions and have satisfactory performance in the presence of various levels of noise and outlier.

Link

Lab Meeting Dec. 8, 2011 (Jim): Execution of a Dual-Object (Pushing) Action with Semantic Event Chains

Title: “Execution of a Dual-Object (Pushing) Action with Semantic Event Chains”
Authors: Aksoy Eren Erdal, Dellen Babette, Tamosiunaite Minija, and Wörgötter Florentin
In IEEE-RAS Int. Conf. on Humanoid Robots, pp.576-583

Abstract:
Here we present a framework for manipulation execution based on the so called “Semantic Event Chain” which is an abstract description of relations between the objects in the scene. It captures the change of those relations during a manipulation and thereby provides the decisive temporal anchor points by which a manipulation is critically defined. Using semantic event chains a model of a manipulation can be learned. We will show that it is possible to add the required control parameters (the spatial anchor points) to this model, which can then be executed by a robot in a fully autonomous way. The process of learning and execution of semantic event chains is explained using a box pushing example

Link