CMU talk: Fast Feature Detection and Stochastic Parameter Estimation of Road Shape using Multiple LIDAR

FRC Seminar

Fast Feature Detection and Stochastic Parameter Estimation of Road Shape using Multiple LIDAR

Kevin Peterson
PhD Student, Robotics Institute, Carnegie Mellon University


Thursday, May 7th, 2009

Developers of autonomous vehicles must overcome significant challenges before these vehicles can operate around human drivers. In urban environments autonomous cars will be required to follow complex traffic rules regarding merging and queuing, navigate in close proximity to other drivers, and safely avoid collision with pedestrians and fixed obstacles near the road. Knowledge of the location and shape of the roadway near then autonomous car is fundamental to these behaviors. While it is tempting to build an /a priori /GPS-registered map of the road network, the possibility for change in the road structure (e.g. new roads, construction, etc.) precludes the use of maps alone. It is therefore necessary to detect and track roads in real-time.

A rich body of work exists in the area of road tracking. Although some early work was performed on unimproved roads, a majority of the available research focuses on paved roads and highways. Additionally, a vast majority of this work focuses on the use of cameras as the single sensing modality. In this talk I will present a framework for road tracking that uses a particle filter to fuse several sources of data including LIDAR and video. The approach enables road tracking on unimproved roads and, because of the flexible nature of the particle filter, can easily be extended to incorporate new forms of data. I will present results from our preparations for the DARPA Urban Challenge.

Speaker Bio: Kevin Peterson’s research focuses on perception techniques for robust autonomous vehicles. Kevin holds a B.S. and M.S. in Electrical and Computer Engineering from Carnegie Mellon University in Pittsburgh, Pennsylvania and is currently pursuing a PhD in Robotics also from Carnegie Mellon University. Kevin has built software systems for many autonomous systems with applications ranging from cave exploration to unexploded ordinance cleanup. Notably, Kevin led Red Team Too, one of CMUs entries in the 2005 DARPA Grand Challenge and was a participant on Tartan Racing, CMUs winning entry in the 2007 Urban Grand Challenge. Since then, he has been applying inverse optimal control techniques to build models of pedestrian motion in structured and unstructured environments.

NTU PhD oral: Mobile Agent Enhanced Service-Oriented Smart Home Architecture and its Human-System Interaction Framework and Algorithm

NTU CSIE PhD oral:

Mobile Agent Enhanced Service-Oriented Smart Home Architecture and its Human-System Interaction Framework and Algorithm

Chao-Lin Wu

Date: May 7, 2009 (Thursday)
Time: 10 am~ 12 noon
Place: CSIE R340

Advisor: Li-Chen Fu

NTU PhD oral: Computer Vision Techniques for Effective Pedestrian Detection

Computer Vision Techniques for Effective Pedestrian Detection

Yu-Ting Chen

Date: May 6, 2009 (Wednesday)
Time: 10 am~12 noon
Place: CSIE R440

Advisors: Chu-Song Chen and Yi-Ping Hung

口試委員:
(1) 校內：洪一平老師、陳祝嵩老師、傅立成老師、王傑智老師
(2) 校外：陳世旺老師、王聖智老師、賴尚宏老師、鍾國亮老師

MIT PhD Thesis: Robust and Efficient Robotic Mapping

MIT PhD Thesis

Title: Robust and Efficient Robotic Mapping
Author: Edwin B. Olson
Date: June 2008

Abstract
Mobile robots are dependent upon a model of the environment for many of their basic functions. Locally accurate maps are critical to collision avoidance, while large-scale maps (accurate both metrically and topologically) are necessary for efficient route planning. Solutions to these problems have immediate and important applications to autonomous vehicles, precision surveying, and domestic robots.
Building accurate maps can be cast as an optimization problem: find the map that is most probable given the set of observations of the environment. However, the problem rapidly becomes difficult when dealing with large maps or large numbers of observations. Sensor noise and non-linearities make the problem even more difficult— especially when using inexpensive (and therefore preferable) sensors.
This thesis describes an optimization algorithm that can rapidly estimate the maximum likelihood map given a set of observations. The algorithm, which iteratively reduces map error by considering a single observation at a time, scales well to large environments with many observations. The approach is particularly robust to noise and non-linearities, quickly escaping local minima that trap current methods. Both batch and online versions of the algorithm are described.
In order to build a map, however, a robot must first be able to recognize places that it has previously seen. Limitations in sensor processing algorithms, coupled with environmental ambiguity, make this difficult. Incorrect place recognitions can rapidly lead to divergence of the map. This thesis describes a place recognition algorithm that can robustly handle ambiguous data.
We evaluate these algorithms on a number of challenging datasets and provide quantitative comparisons to other state-of-the-art methods, illustrating the advantages of our methods.


Link: pdf

CMU talk: Click Chain Model in Web Search

ML Lunch talk:
Speaker: Fan Guo
Date: Monday, May 4, 2009

Title: Click Chain Model in Web Search

Abstract:
Given a terabyte click log, can we build an efficient and effective click model? It is commonly believed that web search click logs are a gold mine for search business, because they reflect users' preference over web documents presented by the search engine. Click models provide a principled approach to inferring user-perceived relevance of web documents, which can be leveraged in numerous applications in search businesses. Due to the huge volume of click data, scalability is a must. I will present the click chain model, which is based on a solid, Bayesian framework. It is both scalable and incremental, perfectly meeting the computational challenges imposed by the voluminous click logs that constantly grow.

Joint work with Chao Liu, Anitha Kannan, Tom Minka, Michael Taylor, Yi-Min Wang and Christos Faloutsos.

NTU talk: AHCBoost: Boosting for Multi-class Classification

Title: AHCBoost: Boosting for Multi-class Classification
Speaker: Prof. Andy Chen-Hai Tsao, National Dong Hwa University
Time: 02:20pm, May 22 (Friday), 2009
Place: Room 102, CSIE building

Abstract:
AdaBoost is one of the important ensemble classifiers developed in the last decade. However, there are difficulties in applying AdaBoost for multi-class classifications. In this study, we introduce the adjustable hyperbolic cosine loss to develop a new boosting algorithm, AHCBoost. Our experiments on benchmark data sets suggest that AHCBoost is very competitive with or even better than the multi-class classifiers SVM and glmBoost. In addition to the fast reduction of training and testing errors, AHCBoost is relatively immune to overfitting and requires little parameter tuning. Some experiments for exploring the potential and limitation of using AHCBoost for ordinal response will also be reported.

Short Biography:

Professional Positions
Department of Applied Math, National Dong Hwa University
* Professor: 2005 to present
* Associate Professor: 1997 to 2005
* Assistant Professor : 1995 to 1997

Institute of Math Statistics, National Chung Cheng University
* Visiting Associate Professor: 1994 to 1995

CMU talk: Learning to Search: Structured Prediction Techniques for Imitation Learning

PhD Thesis Defense:

Learning to Search: Structured Prediction Techniques for Imitation Learning

Nathan D. Ratliff
Carnegie Mellon University

May 01, 2009

Abstract: Modern robots successfully manipulate objects, navigate rugged terrain, drive in urban settings, and play world-class chess. Unfortunately, programming these robots is challenging, time-consuming and expensive; the parameters governing their behavior are often unintuitive, even when the desired behavior is clear and easily demonstrated. Inspired by successful end-to-end learning systems such as neural network controlled driving platforms (Pomerleau, 1989), learning-based "programming by demonstration" has gained currency as a method to achieve intelligent robot behavior. Unfortunately, with highly structured algorithms at their core, it is not clear how to effectively and efficiently train modern robotic systems using classical learning techniques. Rather than redefining robot architectures to accommodate existing learning algorithms, in this thesis I develop learning techniques that leverage the performance of modern robotic components.

My presentation begins with a discussion of a novel imitation learning framework we call Maximum Margin Planning which automates finding a cost function for optimal planning and control algorithms such as A*. In the linear setting, this framework has firm theoretical backing in the form of strong generalization and regret bounds. Further, I have developed practical nonlinear generalizations that are effective and efficient for real-world problems. This framework reduces imitation learning to a modern form of machine learning known as Maximum Margin Structured Classification (Taskar et al. 2005); these algorithms, therefore, apply both specifically to training existing state-of-the-art planners, as well as broadly to solving a range of structured prediction problems of importance in learning and robotics.

In difficult high-dimensional planning domains, such as those found in many manipulation problems, high-performance planning technology remains a topic of much research. I will present some recent work which moves toward simultaneously advancing this technology while retaining the learnability developed above.

I'll demonstrate our algorithms on a range of applications including overhead navigation, quadrupedal locomotion, heuristic learning, manipulation planning, grasp prediction, driver prediction, pedestrian prediction, optical character recognition, and LADAR classification.

link

CMU talk: Camera and LIDAR Fusion for Mapping in Dark Environments

FRC Seminar:

Camera and LIDAR Fusion for Mapping in Dark Environments

Uland Wong
PhD Student, Robotics Institute, Carnegie Mellon University

Thursday, April 30th, 2009

Abstract: Unlit diffuse environments like subterranean voids on earth and planetary surfaces elsewhere are of great interest for robotic exploration and exploitation. These environments pose unique obstacles and constraints, including the necessity for active perception and illumination. However, uniformity of albedo, lack of external lighting and known surface reflectance provide additional assumptions which can be used to enhance 3D-mapping and photographic data collected from robots. This talk presents a method for improving the accuracy of super-resolution point clouds by fusing actively illuminated HDR camera imagery with LIDAR data in dark Lambertian environments. The key approach is shape recovery from estimation of the illumination function and integration in a Markov Random Field (MRF) framework. Experimental results collected from a virtual reconstruction of the Bruceton Research Mine in Pittsburgh, PA are also presented.

NTU talk: Community Discovery in Dynamic, Rich Media Social Networks

Title: Community Discovery in Dynamic, Rich Media Social Networks
Speaker: Yu-Ru Lin, PhD Candidate, Arizona State University.
Time: 3:50pm ~ 5:00pm, Wednesday, March 22, 2009.
Place: Room 111, CSIE building

Abstract: With the rapid proliferation of different types of social media, such as instant messaging (e.g., AIM, MSN, Skype), media sharing sites (e.g., Flickr, YouTube), blogs (e.g., Blogger, WordPress, LiveJournal), wikis (e.g., Wikipedia, PBWiki), microblogs (e.g., Twitter, Jaiku), social networks (e.g., MySpace, Facebook), to mention a few, users routinely produce (e.g. blogs) and consume media (e.g. YouTube) as well as interact with each other through a wide array of functionality provided by various social media. These social media depend largely on implicit communities of online users to deliver value. Identifying and analyzing the dynamics of such latent communities can lead to improved functionality of the social media as well as provide insight into the design of future online collaborative services. The problem is particularly important in the enterprise domain where extracting emergent community structure on enterprise social media, can help in forming new collaborative teams, in expertise discovery, and guide long term enterprise reorganization.

In this talk, I will cover three aspects of community analysis in dynamic, rich media social networks: (1) Community evolution – How do we identify communities in large scale, dynamic social networks, and analyze their structures and evolutions? I will introduce a robust unified approach that discovers communities and captures their evolution with temporal smoothness given by historic community structure. (2) Community summarization – How do we summarize community activities, in order to trace community interests or retrieve community generated content? I will present a summarization framework that characterizes the time-evolving patterns of social activities with associated media objects in a community. (3) Multi-relational communities – How do we discover communities when the social networks exist in a highly connected web of contexts (e.g., social groups, geographic locations, time, etc.)? I will discuss a novel multi-relational non-negative tensor decomposition algorithm that aims to solve this problem. I will also show the effectiveness of these techniques in real world datasets collected from the blogosphere, an enterprise, Flickr, Digg, etc.

Short Biography: Yu-Ru Lin is currently a Ph.D. student in the School of Computing and Informatics at Arizona State University, with a concentration in Arts, Media and Engineering. Her advisor is Dr. Hari Sundaram. Her research interests include problems relating to dynamic multi-relational social network analysis – in particular, community dynamics, social information summarization and representation. Her research focuses on extracting human communities that collaborate around certain topics or media sharing activities. She has proposed non-negative matrix/tensor factorization techniques for analyzing community structures and evolutions in online social networks, as well as time-varying social relational data. Her work has been published in leading international conferences and journals. (Her publication can be found at http://www.public.asu.edu/~ylin56/pub.html.)
She has worked at NEC Labs America and IBM TJ Watson Research Center as a summer intern in 2006, 2007 and 2008. She has received awards including AME Student Excellence Award (2007 and 2008) and IBM PhD Fellowship Award (2009). She holds an M.S. and B.S. degree in Computer Science from National Chiao Tung University, Taiwan.

NTU talk: Mining Geotagged Photos for Semantic Understanding

Title: Mining Geotagged Photos for Semantic Understanding
Speaker: Dr. Jiebo Luo, IEEE Fellow, Senior Principal Scientist with the Kodak Research Laboratories.
Time: 2:30pm ~ 3:40pm, Wednesday, March 22, 2009.
Place: Room 111, CSIE building

Abstract:
Semantic understanding based only on vision cues has been a challenging problem. This problem is particularly acute when the application domain is unconstrained photos available on the Internet or in personal repositories. In recent years, it has been shown that metadata captured with pictures can provide valuable contextual cues complementary to the image content and can be used to improve classification performance. With the recent geotagging phenomenon, an important piece of metadata available with many geotagged pictures is GPS information. We will describe a number of novel ways to mine GPS information in a powerful contextual inference framework that boosts the accuracy of semantic understanding. With integrated GPS-capable cameras on the horizon and geotagging on the rise, this line of research will revolutionize event recognition and media annotation.

Short Biography: Jiebo Luo is a Senior Principal Scientist with the Kodak Research Laboratories in Rochester, NY. He received a B.S. degree and M.S. degree in Electrical Engineering from the University of Science and Technology of China (USTC) in 1989 and 1992, respectively, and a Ph.D. degree in Electrical Engineering from the University of Rochester in 1995. His research interests include signal and image processing, pattern recognition, computer vision, and the related multi-disciplines such as multimedia data mining, biomedical informatics, computational photography, and human-computer interaction. Dr. Luo has authored over 130 technical papers and holds 50 granted US patents. Dr. Luo actively participates in numerous technical conferences, including serving as the chair of the 2008 ACM International Conference on Content-based Image and Video Retrieval (CIVR), an area chair of the 2008 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), a program co-chair of the 2007 SPIE International Symposium on Visual Communication and Image Processing (VCIP), a member of the Organizing Committee of the 2008 ACM Multimedia Conference, 2006 & 2008 IEEE International Conference on Multimedia and Expo (ICME) and 2002 IEEE International Conference on Image Processing (ICIP), and the chair of the IEEE CVPR Workshop on Semantic Learning Application in Multimedia (SLAM) since its inception in 2006. He is the Editor-in-Chief for the Journal of Multimedia (Academy Publisher). Currently, he is also on the editorial boards of the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), the IEEE Transactions on Multimedia (TMM), Pattern Recognition (PR), and Journal of Electronic Imaging (JEI). He is a guest editor for a number of influential special issues, including “Image Understanding for Digital Photos” (PR, 2005), “Real-World Image Annotation and Retrieval” (TPAMI, 2008), “Event Analysis in Video” (IEEE Transactions on Circuits and Systems for Video Technology, 2008), “Integration of Content and Context for Multimedia Management” (TMM, 2009), and “Probabilistic Graphic Models in Computer Vision” (TPAMI, 2009). Dr. Luo is an adjunct professor at Rochester Institute of Technology, as well as the co-dvisor or thesis committee member of many PhD and MS graduate students in various US universities. He is a Kodak Distinguished, a Fellow of SPIE for achievements in electronic imaging and visual communication, and a Fellow of IEEE for contributions to semantic image understanding and intelligent image processing.

Lab Meeting April 20, 2009 (Casey): Avoiding moving outliers in visual SLAM by tracking moving objects

Title: Avoiding moving outliers in visual SLAM by tracking moving objects
Authors: Somkiat Wangsiripitak and David W Murray

In: ICRA 2009

Abstract:

To work at video rate, the maps that monocular SLAM builds are bound to be sparse, making them sensitive to the erroneous inclusion of moving points and to the deletion of valid points through temporary occlusion. This paper describes the parallel implementation of monoSLAM with a 3D object tracker, allowing reasoning about moving objects and occlusion. The SLAMprocess provides the object tracker with information to register objects to the map’s frame, and the object tracker allows the marking of features, either those on objects, or those created by their occluding edges, or those occluded by objects. Experiments are presented to verify the recovered geometry and to indicate the impact on camera pose in monoSLAM of including and avoiding moving features.

[Download]

Lab Meeting April 20, 2009 (Yuchun): An Affective Guide Robot in a Shopping Mall

Title: An Affective Guide Robot in a Shopping Mall
Authors: Takayuki Kanda, Masahiro Shiomi, Zenta Miyashita, Hiroshi Ishiguro and Norihiro Hagita HRI 2009: 173-180

Abstract:
To explore possible robot tasks in daily life, we developed a guide robot for a shopping mall and conducted a field trial with it. The robot was designed to interact naturally with customers and to affectively provide shopping information. It was also designed to repeatedly interact with people to build a rapport; since a shopping mall is a place people repeatedly visit, it provides the chance to explicitly design a robot for multiple interactions. For this capability, we used RFID tags for person identification. The robot was semi- autonomous, partially controlled by a human operator, to cope with the difficulty of speech recognition in a real environment and to handle unexpected situations.

A field trial was conducted at a shopping mall for 25 days to observe how the robot performed this task and how people interacted with it. The robot interacted with approximately 100 groups of customers each day. We invited customers to sign up for RFID tags and those who participated answered questionnaires. The results revealed that 63 out of 235 people in fact went shopping based on the information provided by the robot. The experimental results suggest promising potential for robots working in shopping malls.

[pdf]

CMU talk: Looking without Seeing is in fact Seeing without Knowing -- Insights from Gaze-tracked Change Blindness Studies

Center for the Neural Basis of Cognition Seminar:

Looking without Seeing is in fact Seeing without Knowing
-- Insights from Gaze-tracked Change Blindness Studies

Stella X. Yu
Clare Boothe Luce Assistant Professor
Computer Science @ Boston College
http://www.cs.bc.edu/~syu

Tuesday, April 28

Abstract: Change blindness experiments demonstrate that human vision often neglects certain aspects of the visual scene while attending to others. Using a gaze-tracked flicker paradigm and synthetic images equally rendered in three fundamental features, we explore whether and how innate feature processing might be responsible for the blindness to a change. Our analysis of detection accuracy, detection time, and gaze patterns in this active visual search task reveals distinctive feature extraction, discrimination, and selectivity of size, color, and orientation that underly different behaviours of change blindness. With an array of two-feature stimuli where a single element could change in either feature dimension, we discover that what is changing is sensed long before the subject consciously detects the change, and the change detection task is not accomplished in a single thread of searching for a nonspecific change, but in three separate threads: sensing what the change is, localizing where it might be, and discerning how it is actualized.


Bio: Stella X. Yu got her Ph.D. from the School of Computer Science at Carnegie Mellon University, where she studied robotics at the Robotics Institute and vision science at the Center for the Neural Basis of Cognition. She then went to the University of California, Berkeley to continue her research on computer vision. Her research interests are visual perception and computer vision. Since she joined Boston College, Dr. Yu has been developing an interdisciplinary curriculum and research agenda around art and vision. Her 5-year NSF CAREER proposal, entitled Art and Vision: Scene Layout from Pictorial Cues, was awarded in 2007. Her recent works include image segmentation, object matching, spatial layout categorization and inference, change blindness, and brightness perception.

CMU talk: Multi-robot Coordination for Domains with Intra-path Constraints

CMU FRC Seminar

Multi-robot Coordination for Domains with Intra-path Constraints

E. Gil Jones
PhD Candidate
CMU Robotics Institute

Thursday, April 16nd, 2009

Abstract
Many applications require teams of robots to cooperatively execute complex tasks. Among these domains are those where successful coordination solutions must respect constraints that occur on the intra-path level. This work focuses on multi-agent coordination for disaster response with intra-path constraints, a compelling application that is not well addressed by current coordination methods. In this domain a group of fire trucks agents attempt to address a number of fires that are occurring throughout a city in the wake of a large-scale disaster. The disaster has not only caused fires but has also caused many city roads to be blocked by debris, making them impassable; bulldozer robots also operating in the domain can clear the debris. The coordination solution must determine not only a task allocation but also what routes the fire trucks should take given the intra-path precedence constraints and which bulldozers should be assigned to satisfy those constraints. This talk will focus on two main techniques for determining multi-robot coordination solutions for domains with intra-path constraints. The first technique uses tiered auctions, a novel market-based method. The second technique uses centralized genetic algorithms. The approaches are compared in terms of solution quality and computation time in a simulated disaster response domain.

Bio
Gil is a fifth year Ph.D. student at the Robotics Institute, and is co-advised by Bernardine Dias and Tony Stentz. He received his BA in Computer Science from Swarthmore College in 2001, and spent two years as a software engineer at Bluefin Robotics - manufacturer of autonomous underwater vehicles - in Cambridge, Mass.

CMU talk: Inferring Object Attributes

RI Seminar
Inferring Object Attributes

Derek Hoiem
Assistant Professor, University of Illinois at Urbana Champaign

April 10, 2009

Abstract: Ultimately, the goal of computer vision is to make useful inferences from imagery, and a big part of that is knowing something about the properties of nearby objects. In this talk, I'll describe our recent work on learning to identify object attributes, such as parts, materials, or shape, from images in a way that generalizes to new object categories. The tricky part is training classifiers that really predict the intended attribute, and not ones that are correlated through familiar object categories. Once we can predict attributes, we can say what is unusual about an object and more easily learn to recognize new objects Sometimes we can even recognize new object categories from a purely verbal description (e.g., a goat has four legs, horns, and is furry).

This work is with Ali Farhadi, Ian Endres, and David Forsyth at UIUC.

Speaker Bio.: Derek Hoiem is a new assistant professor at University of Illinois at Urbana Champaign. Derek researches object recognition, segmentation, 3d reconstruction from images, and other aspects of computer vision that are related to scene understanding. He recently (2007) graduated from the Robotics Institute under the tutelage of Alyosha Efros and Martial Hebert and looks forward to visiting. By request, Derek will share a little of his perspective in transitioning from being a grad student at CMU to a professor at UIUC.

CMU talk: Fundamental Limits of Imaging in Scattering Media

VASC Seminar
Monday, April 13, 2009

Fundamental Limits of Imaging in Scattering Media
Tali Treibitz
Technion

Abstract:
Scattering media exist in bad weather, liquids, biological tissue and even solids. Images taken into scattering media suffer from resolution loss photometric as well as geometric problems. In addition, high noise levels impose resolution limits, even if there is no blur. These problems inhibit vision in such media. In the talk I give an overview of our contributions in this subject:
- Resolution limits imposed by noise
- Limits in polarization-based dehazing
- Geometry limits: The non-single viewpoint nature of imaging systems looking into water through a flat glass.

Bio:
Tali Treibitz received her BA degree in computer science from the Technion-Israel Institute of Technology in 2001. She is currently a Ph.D. candidate in the department of Electrical Engineering, Technion. Her research involves physics-based computer vision. She is also an active PADI open water scuba instructor.

CMU talk: Discourse Structure from Topic Models in Text and Video

CMU ML Lunch
Speaker: Jacob Eisenstein (UIUC)
Venue: NSH 1507
Date: Monday, April 13, 2009

Title:
Discourse Structure from Topic Models in Text and Video

Abstract:
This talk describes how latent topic models can be used to discover discourse structure in unannotated text and video. Linguists have long believed that discourse topic shifts are marked by changes in the distribution of lexical items. This idea is called lexical cohesion, and can be formalized in a latent topic model, yielding substantial performance gains over previous heuristic approaches. More importantly, this Bayesian setting permits several interesting extensions:
(1) Explicit cue phrases for topic transitions are clearly relevant for segmentation, but can not be handled by previous unsupervised methods. I'll show how such cue phrases can be discovered without annotation and incorporated to improve segmentation.
(2) Topic segmentation can be applied to multimedia data by searching for self-similarity in visual communication. I'll present a topic segmenter for conversational speech that integrates lexical and gestural cohesion.
(3) Hierarchical structure can be discovered by modeling lexical cohesion as a multi-scale phenomenon, in which some words are governed by low-level subtopics, and others by the high-level topics. Inference is performed jointly across scale-levels, improving on greedy top-down approaches.

Overall, these extensions outperform the state-of-the-art on several tasks, and point the way to more comprehensive analysis of discourse structure through hierarchical Bayesian models.

BIO: Jacob Eisenstein is a Beckman Postdoctoral Fellow at the University of Illinois. He completed his doctorate at MIT in 2008 under the supervision of Regina Barzilay and Randall Davis. His thesis, titled "Gesture in Automatic Discourse Processing," won the 2008 George M. Sprowls award for best Doctoral theses in Computer Science at MIT. Working in the domain of computational linguistics, Jacob's research focuses on applying state-of-the-art structured learning techniques to discourse processing and visual communication.

Lab Meeting April 13,2009 (Jimmy): Mutual Localization in a Team of Autonomous Robots using Acoustic Robot Detection

Title: Mutual Localization in a Team of Autonomous Robots using Acoustic Robot Detection

Authors: David Becker and Max Risler

In: RoboCup International Symposium 2008

Abstract

In order to improve self-localization accuracy we are exploring ways of mutual localization in a team of autonomous robots. Detecting team mates visually usually leads to inaccurate bearings and only rough distance estimates. Also, visually identifying teammates is not possible. Therefore we are investigating methods of gaining relative position information acoustically in a team of robots.

The technique introduced in this paper is a variant of code-multiplexed communication (CDMA, code division multiple access). In a CDMA system, several receivers and senders can communicate at the same time, using the same carrier frequency. Well-known examples of CDMA systems include wireless computer networks and the Global Positioning System, GPS. While these systems use electro-magnetic waves, we will try to adopt the CDMA principle towards using acoustic pattern recognition, enabling robots to calculate distances and bearings to each other.

First, we explain the general idea of cross-correlation functions and appropriate signal pattern generation. We will further explain the importance of synchronized clocks and discuss the problems arising from clock drifts.

Finally, we describe an implementation using the Aibo ERS-7 as platform and briefly state basic results, including measurement accuracy and a runtime estimate. We will briefly discuss acoustic localization in the specific scenario of a RoboCup soccer game.

[pdf]

Lab Meeting April 13,2009 (Gary) Subtle facial expression recognition using motion magnification

Title: Subtle facial expression recognition using motion magnification
Authors: Sungsoo Park, Daijin Kim

Pattern Recognition Letters
Volume 30, Issue 7, 1 May 2009, Pages 708-716

Abstract
This paper proposes a novel method for subtle facial expression recognition that uses motion magnification to transform subtle expressions into corresponding exaggerated ones. Motion magnification consists of four steps: First, active appearance model (AAM) fitting extracts 70 facial feature points in the face image sequence. Second, the face image sequence is aligned using the three feature points (two eyes and nose tip). Third, the motion vectors of 27 feature points are estimated using the feature point tracking method. Finally, exaggerated facial expressions are obtained by magnifying the motion vectors of the 27 feature points. After motion magnification, the exaggerated facial expressions are recognized as follows: first, the shape and appearance features are obtained by projecting the exaggerated facial expression image to the AAM shape and appearance model. Second, support vector machines (SVM) are used to classify shape and appearance features. Experimental results show that proposed subtle facial recognition rate is 88.125% for the 80 facial expression images in the SFED2007 database.


link

NTU talk: Skill Learning for Humanoid Robots

NTU CSIE Seminar
Time: 2:20 PM, May 8, 2009 
Place: NTU CSIE

Skill Learning for Humanoid Robots

Hsien-I Lin
School of Electrical and Computer Engineering
Purdue University
West Lafayette, IN 47907-2035
sofin@purdue.edu

Abstract:
Recent advances in human-centered robots such as humanoid robots are driven by the projection that these robots will have a place in our society and in our daily life activities as assistive robots. Endowing these humanoid robots with the ability of skill learning will enable them to be versatile and skillful in performing various tasks. The problem of transferring human skills to humanoid robots raises tremendous research interest in studying human and robot motor skills. Our current research aims at developing a quantitative measure of robot motor capability of a humanoid root motor system for the application of transferring human skills to a humanoid robot.

We propose to employ an information-theory-based method to quantitatively represent the robot motor capability by a pseudo index of motor performance. This pseudo index of motor performance is derived from kinematics, dynamics, and control with the speed-accuracy constraint taken into consideration. With the speed-accuracy constraint, we are able to optimize the motor performance of a robot to accomplish a task by satisfying the task spatial and temporal constraints. Computer simulations and experimental work were performed on a 6 DOF PUMA robot to validate the performance of the proposed approach in measuring the robot motor capability of a root motor system.

Bio: Hsien-I Lin received the B.S. and M.S. degrees in Electrical and Control Engineering from National Chiao Tung University in 1997 and 1999, respectively, and he is currently a Ph.D. candidate in Electrical and Computer Engineering at Purdue University, West Lafayette, Indiana. Before beginning his academic career, he worked for the VIA technologies, Inc., Taipei, Taiwan during 2001-2003, after which he was a research assistant of the Department of Bio-Industrial Mechatronics Engineering at National Taiwan University during 2003-2004. Since then, he has been a research assistant of the Department of Electrical and Computer Engineering at Purdue University. His research interests are in the areas of human-robot interaction with emphasis on robot skill learning, intelligent systems, and neuro-fuzzy networks.

CMU talk: Fourier Theoretic Probabilistic Inference over Permutations

Speaker: Jonathan Huang (RI @ CMU)

Venue: NSH 1507

Date: Monday, April 6, 2009

Time: 12:00 noon

Title:

Fourier Theoretic Probabilistic Inference over Permutations

Abstract:

Permutations are ubiquitous in many real-world problems, such as voting, ranking, and data association. Representing uncertainty over permutations, however, is challenging, since there are $n!$ possibilities, and common factorized probability distribution representations, such as graphical models, are inefficient due to the mutual exclusivity constraints that are typically associated with permutations.

I will talk about a recent approach for probabilistic reasoning with permutations based on the idea of approximating distributions using their low-frequency Fourier components. Maintaining the appropriate set of low-frequency Fourier terms corresponds to maintaining matrices of simple marginal probabilities which summarize the underlying distribution. Using these intuitions, I will show how to derive the Fourier coefficients of a variety of probabilistic models which arise in practice and that many useful models are either well-approximated or exactly represented by low-frequency (and in many cases, sparse) Fourier coefficient matrices.

In addition to showing that Fourier representations are both compact and intuitive, I will show how to cast common probabilistic inference operations in the Fourier domain, including marginalization, conditioning on evidence, and factoring based on probabilistic independence.

From the theoretical side, our work has tackled several problems in understanding the consequences of the bandlimiting approximation. On this front, I will present illuminating results about the nature of error propagation in the Fourier domain and propose methods for mitigating their effects.

Finally I will demonstrate the approach on several real datasets and show that our methods, in addition to being well-founded theoretically, are also scalable and provide superior results in practice.

joint work with Carlos Guestrin, Leonidas Guibas and Xiaoye Jiang

News: A Big-Screen Display as Mobile as Your Phone

Spectrum Video: A Big-Screen Display as Mobile as Your Phone
Projector phones give us the first glimpse at a future where device size and display size are independent. The companies behind liquid crystal on silicon, digital light processing, and laser projection technologies each think they have a superior technology, but watch the video and decide for yourself.
View Now!

I just can not forget our previous work on the camera-projector systems. -Bob

CMU talk: Next Generation Map Making

Next Generation Map Making

Xin Chen
NAVTEQ

VASC Seminar
Monday, April 6, 2009

Abstract:
NAVTEQ is a leading global provider of digital map data. NAVTEQ maps drive most in-vehicle navigation systems, the top routing web sites, and the leading brands of wireless navigation devices. NAVTEQ continues to enhance the technologies used for collecting, analyzing, and delivering new content to a wide range of users and devices. Dr. Chen will discuss NAVTEQ's perspective on the hardware and software systems required to automatically create and maintain a navigable map through the use of high-end mobile data collection sensors and computer vision techniques. Dr. Chen will also present numerous research efforts based on video and LIDAR data collection as well as various challenging problems related to automatic feature extraction for mapping and navigation.

Bio:
Dr. Xin Chen currently works as a senior researcher in the Research and Emerging Technologies Department of NAVTEQ Corporation. His recent research efforts have concentrated on computer vision, pattern recognition and image processing of video, aerial photo and LIDAR. He received a Ph.D. in Computer Science and Engineering from the University of Notre Dame. His research at Notre Dame focused on biometrics, including infrared, 2D, and 3D face recognition.

CMU talk: 3-D Stereo Vision SLAM for Autonomous Mapping and Navigation (live demonstration)

FRC Seminar

3-D Stereo Vision SLAM for Autonomous Mapping and Navigation (live demonstration)

Pearse Ffrench
Project Manager and Senior Technical Staff Member
Vision Robotics
San Diego, CA


Thursday, April 2nd, 2009

Vision Robotics has developed a 3-D stereo vision SLAM, visual odometry, object identification, and tracking system for autonomous robots. The current applications include an autonomous canister style vacuum cleaner and a sensor system for military UGV's. The presentation will describe the 3-D stereo vision SLAM algorithms including the stereo vision processing, particle based localization, and 3-D map generation. It will conclude with a demonstration of the 3-D stereo vision SLAM sensor system on a prototype Autonomous Mapping Vehicle (AMV) being developed for SPAWAR System Center San Diego. The current prototype uses a Segway RMP-50 as a base platform, has four stereo camera pairs, and maps at over 100 square feet per minute with speeds over 2 mph. An external link via 802.11N to a laptop provides for start/stop/pause control and real-time display of the map and pose data. The AMV will autonomously move throughout an area, and when it has finished exploring the complete area, it will return to the starting location.

Speaker Bio
Pearse received the B.S. in electrical engineering from Massachusetts Institute of Technology in 1989 and the M.S. in electrical engineering from the University of California, San Diego in 1994. Pearse Ffrench is a project manager and senior technical staff member at Vision Robotics. He has been developing 3-D stereo vision SLAM and tracking algorithms at Vision Robotics for the past eight years. Previously, he was the founder and CTO of Digital Transport Systems, a communications test equipment company which was acquired by Wavetek Wandel Golterman in 1998. From 1989 to 1994, he was the lead engineer for Advanced Processing Laboratories on several Navy projects involving underwater acoustics and signal processing.

Lab Meeting April 6, 2009 (Any): RANSAC: An Historical Perspective

In past few years, robust statistical methods have significantly contributed to the advances made in computer vision. In particular, the class of random-sample consensus, or RANSAC type algorithms for the solution of problems in robust estimation has been key to progress.

The fitting problem is approached in the opposite way from most previous techniques, such as the least squares approaches. Instead of averaging all the measurements and then trying to throw out bad ones, the smallest number of measurements is used to compute a model’s unknown parameters and then evaluated the instantiated model by counting the number of consistent samples.

I will review the RANSAC technique and some of its variants/applications, as well as the future research directions.

CVPR 2006 Workshop: 25 Years of RANSAC - http://cmp.felk.cvut.cz/ransac-cvpr2006/

Lab Meeting April 6, 2009 (Shao-Chen): Learning for Control from Multiple Demonstrations

Title: Learning for Control from Multiple Demonstrations
Authors: Adam Coates, Pieter Abbeel, Andrew Y. Ng

International Conference on Machine Learning(ICML), Best Application Paper Award

Abstract
We consider the problem of learning to follow a desired trajectory when given a small num-
ber of demonstrations from a sub-optimal expert. We present an algorithm that (i) extracts the—initially unknown—desired trajectory from the sub-optimal expert’s demonstrations and (ii) learns a local model suitable for control along the learned trajectory.
We apply our algorithm to the problem of autonomous helicopter flight. In all cases, the autonomous helicopter’s performance exceeds that of our expert helicopter pilot’s demonstrations. Even stronger, our results significantly extend the state-of-the-art in autonomous helicopter aerobatics. In particular, our results include the first autonomous tic-tocs, loops and hurricane, vastly superior performance on previously performed aerobatic maneuvers (such as in-place flips and rolls), and a complete airshow, which requires autonomous transitions between these and various other maneuvers.

Link

CMU talk: Uncovering, understanding, and predicting links

CMU ML Lunch talk
Speaker: Jonathan Chang (CS@Princeton)
Date: Monday, March 30, 2009

Title:
Uncovering, understanding, and predicting links

Abstract:
Linked data are ubiquitous. Friends are connected to friends in social networks, genes interact with genes in biological networks, and papers cite other papers in citation networks. Uncovering, understanding, and making predictions using such links is of critical importance in unlocking and applying this data.

In this talk I will present probabilistic models for these tasks. Our approaches leverage the machinery of topic models, such as latent Dirichlet allocation (LDA), which have been successfully employed for a variety of applications. However, topic models typically make naive assumptions about the independence of documents. By breaking these naive assumptions, we can develop novel models that give insights into the latent structure of networks of documents. By using efficient variational inference, I will show that these models can make accurate predictions and extract new link information from free text.

CMU talk: Observing and Interpreting Everyday Human Activities

VASC Seminar
Monday, March 30, 2009

Observing and Interpreting Everyday Human Activities
Jan Bandouch, Ph.D. Student
Moritz Tenorth, Ph.D. Student
TU Munich, Germany

Abstract:
We will present and discuss an integrated approach for the automated perception, interpretation and analysis of human activities of daily life, with an emphasis on everyday manipulation tasks. During the first half of this talk, we will describe a markerless motion capture system that we use to acquire joint-angle representations of human full-body motions at high accuracy. The system is capable of tracking arbitrary, previously unobserved motions using a sophisticated hierarchical sampling strategy for recursive Bayesian estimation that combines partitioning with annealing strategies to enable efficient search in the presence of many local maxima. A simple yet effective appearance model is used to implicitly deal with occlusions and to reduce the influence of objects and dynamic parts of the environment. We will then show in the second half of our talk how to infer probabilistic, hybrid (continuous/discrete), low-dimensional, and hierarchical models of the observed activity. These models can be used to answer queries about the observed activity such as the following ones: Which activity was performed? Which hand trajectory did the human use for taking a plate out of the overhead cupboard? What was different compared to the normal execution of this activity? The integration of these activity models into a knowledge-based framework allows for the association of the observed activity with encyclopedic, commonsense, and naive physics knowledge and for querying the system in an abstract, symbolic way.

Bio:
Jan Bandouch is a PhD student in the Intelligent Autonomous Systems Group of Prof. Michael Beetz at the TU Munich, Germany. His current research interest is in Computer Vision and Robotics, where he is working on techniques for markerless human motion capture in typical human living environments. The intended areas of application are activity and intention recognition in smart environments, motion analysis and ergonomic studies. He obtained his Diploma (equivalent to M.Sc.) in Computer Science from the TU Munich in 2005. Homepage: http://www9.cs.tum.edu/people/bandouch

Moritz Tenorth is a PhD student in the Intelligent Autonomous Systems Group of Prof. Michael Beetz at the TU Munich, Germany. His research interests include grounded knowledge representations which integrate information from web sources, observed sensor data and data mining techniques, and their applications to knowledge-based action interpretation and robot control. He studied Electical Engineering in Aachen and Paris and obtained his Diploma degree (equivalent to M.Eng.) in 2007 from the RWTH Aachen. Homepage: http://www9.cs.tum.edu/people/tenorth

CMU talk: Human Pose and Motion, Challenges and Physics-based Models

The VASC/DRP morning seminar series.

Title: Human Pose and Motion, Challenges and Physics-based Models

Leonid Sigal, University of Toronto

Date: Tuesday 3/31

Abstract:
Recovery and analysis of human pose and motion from video is the key enabling technology for a broad spectrum of applications, in and outside of computer science; including applications in HCI, biometrics, biomechanics and computer graphics. Despite years of research, the general problem of tracking a person in an unconstrained environment, particularly from monocular observations, remains challenging. In this talk I will describe the basic building blocks and challenges of the articulated human pose estimation and tracking, as well as my contributions to the various aspects of this problem and the field in recent years. I will particularly focus on the new and unique class of models that incorporate physic-based predictions and simulation into the inference process. Physics plays an important and intricate role in characterizing, describing and predicting human motion. The key benefit of using physics-based models or priors for tracking is the improved realism in the recovered motions, as well as enhanced ability to deal with weak image observations and diverse environmental interactions. Newtonian physics, in these models, approximates the rigid-body dynamics of the body in the environment through the application and integration of forces. Since the motion of the body is intimately tied with the environment, the use of such models also allows one to start reasoning about the geometry and physical properties of the environment as a whole (e.g. orientation and compliance of ground). This work is part of joint projects with colleagues at Brown University and University of Toronto.

Biography:
Leonid Sigal is a postdoctoral fellow in the Department of Computer Science at University of Toronto. He received his Ph.D. in computer science from Brown University (2007); his M.S. from Brown University (2003); his M.A. from Boston University (1999); and his B.Sc. degrees in Computer Science and Mathematics from Boston University (1999). From 1999 to 2001, he worked as a senior vision engineer at Cognex Corporation, where he developed industrial vision applications for pattern analysis and verification. In 2002, he spent a semester as a research intern at Siemens Corporate Research (SCR) working on autonomous obstacle detection and avoidance for vehicle navigation. During the summers of 2005 and 2006, he worked as a research intern at Intel Applications Research Lab (ARL) on human pose estimation and tracking. His work received the Best Paper Award at the Articulate Motion and Deformable Objects Conference in 2006 (with Prof. Michael J. Black). Dr. Sigal's research interests mainly lie in the areas of computer vision and machine learning, but also borderline fields of computer graphics, psychology and humanoid robotics. He is particularly interested in statistical models for problems of visual inference, including human motion recovery and analysis, graphical models, probabilistic and hierarchical inference.

CMU talk: Empowering switching linear dynamic systems with higher-order temporal structure

The VASC/Disney Research Pittsburgh seminar

Title: Empowering switching linear dynamic systems with higher-order temporal structure

Sangmin Oh, Georgia Institute of Technology

Date: Friday 3/27

Abstract:
Automated analysis of temporal data is a task of utmost importance for intelligent machines. For example, ubiquitous computing systems need to understand the intention of humans from the stream of sensory information, and health-care monitoring systems can assist patients and doctors by providing automatically annotated daily health reports. Moreover, a huge amount of multimedia data such as videos await to be analyzed and indexed for search purposes, while scientific data such as recordings of animal behavior and evolving brain signals are being collected in the hope to deliver a new scientific discovery about life.

In this talk, we will describe a class of newly developed time-series models in Bayesian network formulation. In particular, we will focus on the extensions of switching linear dynamic systems (SLDSs) with higher-order temporal structure and inference methods thereof. SLDSs have been used to model continuous multivariate temporal data under the assumption that the characteristics of complex temporal sequences can be captured by Markov switching between a set of simpler primitives which are linear dynamic systems (LDSs). In particular, we will focus on the extensions of SLDSs which are developed to address problems such as continuous labeling, robust labeling for data with systematic global variations, and hierarchical labeling.

First, we will present a data-driven MCMC inference method for SLDS model. The distinctive characteristic of this approach is that it turns heuristic labeling methods into data-driven proposal distributions of MCMC where the outcome results in a principled approximate inference method. In other words, it is a methodology to turn a novice into an expert. We show the resulting MCMC method for SLDSs where an inference problem is now solved which could not be addressed efficiently by Gibbs sampling previously.

Second, parametric SLDSs (P-SLDSs) explicitly model the global parameters which induce systematic temporal and spatial variations of data. The additional structure of PSLDSs allows us to conduct the global parameter quantification task which could not be addressed by standard SLDSs previously in addition to providing more accurate labeling ability.

Third, segmental SLDSs (S-SLDSs) provide the ability to capture descriptive duration models within LDS regimes. The encoded duration models are more descriptive than the exponential duration models induced within the standard SLDSs and allow us to avoid the severe problem of over-segmentations and demonstrate superior labeling accuracy.

Finally, we introduce hierarchical SLDSs (H-SLDSs), a generalization of standard SLDSs with hierarchic Markov chains. H-SLDSs are able to encode temporal data which exhibits hierarchic structure where the underlying low-level temporal patterns repeatedly appear in different higher level contexts. Accordingly, H-SLDSs can be used to analyze temporal data at multiple temporal granularities, and provide the additional ability to learn a more complex H-SLDS model easily by combining underlying H-SLDSs.

The developed SLDS extensions have been applied to two real-world problems. The first problem is to automatically analyze the honey bee dance dataset where the goal is to correctly segment the dance sequences into different regimes and parse the messages about the location of food sources embedded in the data. We show that a combination of the P-SLDS and S-SLDS models has demonstrated improved labeling accuracy and message parsing results. The second problem is to analyze the wearable exercise data where we aim to provide an automatically generated exercise record at multiple temporal resolutions.

Bio:
Sangmin is a PhD candidate in Collge of Computing at Georgia Institute of Technology. He received his BS in computer science with cum laude from Seoul National Univ, in 2003. During his PhD thesis work, Sangmin has focussed on developing time-series models to address problems such as continuous labeling, robust labeling for data with systematic global variations, and hierarchical labeling, where he published his work at major conferences and journals in computer vision and AI. Additionally, he worked on problems in robotics, signal processing, and graphics, where he co-authored several academic publications. He was a recipient of Samsung Lee Kun Hee fellowship from '03 to '07. His research interests include computer vision, machine learning, robotics, computer graphics, data mining, time-series modeling and computational linguistics.

Lab Meeting March 23rd, 2009 (Jeff): Sensor fusion framework of my work

Abstract:

I will organize the sensor fusion framework about my current work.

2008 AI's 10 to Watch

IEEE Intelligent Systems is pleased to announce the 2008 edition of the “AI’s 10 to Watch” list—the 10 top researchers in the magazine’s areas of coverage who have received their PhDs in the past couple of years. We solicited nominees from numerous sources, including the AAAI fellows, the chairs of major CS departments around the world, a number of mailing lists in this area, and an open call in our magazine and other public forums. We received a large number of qualified nominees from around the world. A committee from our advisory and editorial boards winnowed this down to the top 10. Read the full article.

CMU talk:Material Recognition By Humans and Machines

VASC Seminar Series

Material Recognition By Humans and Machines
Lavanya Sharan
MIT
2:00pm, Friday, March 20th


Abstract:
We can easily tell if a spoon is made of stainless steel or plastic, if a shirt is clean or if food is fresh. These judgments of material appearance are ubiquitous. We use our material perception abilities to decide where to step on an icy sidewalk, which items to pick in the fresh produce aisle, and if a rash requires a trip to the doctor. In spite of the importance of these judgments, little is known about material recognition in the fields of human vision or computer vision.

We have studied human material judgments on real world photographs by asking observers questions like "Is that object made of paper or plastic?" or "Are those flowers fake or real?". We find that observers can recognize materials very well, even when images are presented very fast (40 millisecond/image). This performance was robust to low-level image degradations like removal of color, blurring and inversion of contrast polarity, suggesting that low-level information is not crucial for observers.

What do these results imply for machine vision systems? We evaluated the performance of simple classifiers based on low-level image features (e.g. jet-like features, SIFT) at the same material categorization task that humans did. We find that low-level features are not sufficient for categorizing materials on our data set suggesting a parallel with the results from human experiments. We conclude that there is rich territory to be explored both by computer vision and human vision researchers for this problem.

Bio:
Lavanya Sharan is a final year graduate student working with Ted Adelson at MIT. Her research interests lie at the intersection of human vision and computer vision, especially in the domain of material recognition. She is interested in understanding how humans can recognize the materials that objects are made of and how to make computer vision systems that can do the same. Lavanya received her M.S. degree in Computer Science from MIT in 2005 and her undergraduate training in Electrical Engineering from IIT Delhi in 2003.

Robot responds to human gestures

Robot responds to human gestures

A Brown University team has developed a robot that can recognize human gestures in multiple environments, despite changes in lighting and depth. The team recorded a video as a proof of concept, demonstrating a robot equipped with continuous tracks, sensors, and a CSEM SwissRanger imaging camera. The robot recognizes its human controller by creating a silhouette that it focuses on as a human cutout, ignoring other environmental input. It follows humans about three feet behind, and moves in reverse when its controller walks towards it.

Wag The Robot? Robot Responds To Human Gestures

ScienceDaily (2009-03-12) -- Researchers have demonstrated how a robot can follow human gestures in a variety of environments -- indoors and outside -- without adjusting for lighting. The achievement is an important step forward in the quest to build fully autonomous robots as partners for human endeavors. ... > read full article

CMU talk: Computational Study Of Nonverbal Social Communication

Special VASC Seminar
Thursday, March 19, 2009

Computational Study Of Nonverbal Social Communication
Louis-Philippe Morency
USC Institute for Creative Technologies

Abstract:
The goal of this emerging research field is to recognize, model and predict human nonverbal behavior in the context of interaction with virtual humans, robots and other human participants. At the core of this research field is the need for new computational models of human interaction emphasizing the multi-modal, multi-participant and multi-behavior aspects of human behavior. This multi-disciplinary research topic overlaps the fields of multi-modal interaction, social psychology, computer vision, machine learning and artificial intelligence, and has many applications in areas as diverse as medicine, robotics and education.

During my talk, I will focus on three novel approaches to achieve efficient and robust nonverbal behavior modeling and recognition: (1) a new visual tracking framework (GAVAM) with automatic initialization and bounded drift which acquires online the view-based appearance of the object, (2) the use of latent-state models in discriminative sequence classification (Latent-Dynamic CRF) to capture the influence of unobservable factors on nonverbal behavior and (3) the integration of contextual information (specifically dialogue context) to improve nonverbal prediction and recognition.

Bio:
Dr. Louis-Philippe Morency is currently research professor at USC Institute for Creative Technologies where he leads the Nonverbal Behaviors Understanding project (ICT-NVREC). He received his Ph.D. from MIT Computer Science and Artificial Intelligence Laboratory in 2006. His main research interest is computational study of nonverbal social communication, a multi-disciplinary research topic that overlays the fields of multi-modal interaction, computer vision, machine learning, social psychology and artificial intelligence. He developed "Watson", a real-time library for nonverbal behavior recognition and which became the de-facto standard for adding perception to embodied agent interfaces. He received many awards for his work on nonverbal behavior computation including three best-paper awards in 2008 (at various IEEE and ACM conferences). He was recently selected by IEEE Intelligent Systems as one of the "Ten to Watch" for the future of AI research.

Lab Meeting March 23, 2009 (Alan): Experimental Analysis of Sample-Based Maps for Long-Term SLAM

Title: Experimental Analysis of Sample-Based Maps for Long-Term SLAM
Authors: Peter Biber, Tom Duckett

The International Journal of Robotics Research, Vol. 28, No. 1, 20-33 (2009)

Abstract
This paper presents a system for long-term SLAM (simultaneous localization and mapping) by mobile service robots and its experimental evaluation in a real dynamic environment. To deal with the stability-plasticity dilemma (the trade-off between adaptation to new patterns and preservation of old patterns), the environment is represented by multiple timescales simultaneously (five in our experiments). A sample-based representation is proposed, where older memories fade at different rates depending on the timescale and robust statistics are used to interpret the samples. The dynamics of this representation are analyzed in a five-week experiment, measuring the relative influence of short- and long-term memories over time and further demonstrating the robustness of the approach.

Link: local copy

CMU talk: Imitation Learning for Autonomous Navigation in Complex Natural Terrain

Imitation Learning for Autonomous Navigation in Complex Natural Terrain

David Silver, PhD Candidate
Robotics Institute
Carnegie Mellon University

Thursday, March 19th, 2009


Abstract:
Rough terrain autonomous navigation continues to pose a challenge to the robotics community. Robust navigation by a mobile robot depends not only on the individual performance of perception and planning systems, but on how well these systems are coupled. When traversing rough terrain, this coupling (in the form of a cost function) has a large impact on robot performance, necessitating a robust design. This talk presents the application of imitation learning to this task. Using expert examples of proper navigation behavior, mappings from both online and offline perceptual data to planning costs are learned. Experimental results are presented from the Crusher autonomous navigation platform, demonstrating a benefit to autonomous performance as well as a decrease in programmer interaction.

Lab Meeting March 16th (swem): Inverse Depth to Depth Conversion for Monocular SLAM(ICRA2007)

Authors:Javier Civera, Andrew J. Davison, J.M.M Montiel

Robotics and Automation, 2007 IEEE International Conference on
10-14 April 2007
Page(s): 2778-2783

Abstract:
Recently it has been shown that an inverse depth
parametrization can improve the performance of real-time
monocular EKF SLAM, permitting undelayed initialization of
features at all depths. However, the inverse depth parametrization
requires the storage of 6 parameters in the state vector for
each map point. This implies a noticeable computing overhead
when compared with the standard 3 parameter XYZ Euclidean
encoding of a 3D point, since the computational complexity of
the EKF scales poorly with state vector size.
In this work we propose to restrict the inverse depth
parametrization only to cases where the standard Euclidean
encoding implies a departure from linearity in the measurement
equations. Every new map feature is still initialized using the
6 parameter inverse depth method. However, as the estimation
evolves, if according to a linearity index the alternative
XYZ coding can be considered linear, we show that feature
parametrization can be transformed from inverse depth to XYZ
for increased computational efficiency with little reduction in
accuracy.
We present a theoretical development of the necessary
linearity indices, along with simulations to analyze the influence
of the conversion threshold. Experiments performed with with a
30 frames per second real-time system are reported. An analysis
of the increase in the map size that can be successfully managed
is included.

Link: http://webdiis.unizar.es/~jcivera/papers/civera_etal_icra07.pdf
Video: http://webdiis.unizar.es/~jcivera/videos/civera_etal_icra07.mp4

CMU talk: Beyond Nouns and Verbs: Learning Visually Grounded Stories of Images and Videos using Language and Vision Abhinav Gupta

VASC Seminar
Monday, March 16, 2009

Beyond Nouns and Verbs: Learning Visually Grounded Stories of Images and Videos using Language and Vision
Abhinav Gupta
University of Maryland, College Park

Abstract:
In this talk, I will present our recent work on exploring synergy between language and vision for learning visually grounded contextual structures. Our work departs from the traditional view to visual and contextual learning where individual detectors and relationships are learned separately. Our work focuses on simultaneous learning of visual appearance and contextual models from richly annotated, weakly labeled datasets. In the first part of the talk, I will show how rich annotations can be utilized to constrain the learning of visually grounded models of nouns, prepositions and comparative adjectives from weakly labeled data. I will also show how visually grounded models of prepositions and comparative adjectives can be utilized as contextual models for scene analysis.

In the second part, I will present storyline models for interpretation of videos. Storyline models go beyond pair-wise contextual models and represent higher order constraints by allowing only a few and finite number of possible action sequences (stories). Visual inference using storyline models involve inferring the "plot" of the video (sequence of actions) and recognizing individual activities in the plot. I will also present an iterative approach to learn visually grounded storyline models from video and linguistic information provided in captions.

Bio:
Abhinav Gupta is a doctoral candidate in the Department of Computer Science at University of Maryland, College Park. He received MS in Computer Science from University of Maryland in 2007 and BTech in Computer Science and Engineering from Indian Institute of Technology, Kanpur, in 2004. His research focuses on visually grounded semantic models, and how language and vision can be exploited to learn such models. His other research interests include combining multiple cues, probabilistic graphical models, human body tracking and camera networks. He is also a recipient of the University of Maryland Dean's Fellowship for excellence in research.

CMU talk: Supervised and relational topic models

Supervised and relational topic models
David Blei
Princeton University

Abstract:

A surge of recent research in machine learning and statistics has developed new techniques for finding patterns of words in document collections using hierarchical probabilistic models. These models are called "topic models" because the discovered word patterns often reflect the underlying topics that permeate the documents. Topic models also naturally apply to data such as images and biological sequences.

In this talk I will review the basics of topic modeling, and discuss some recent extensions: supervised topic modeling and relational topic modeling. Supervised topic models allow us to use topics in a setting where we seek both exploratory and predictive power. Relational topic models---which are built on supervised topic models---consider documents interconnected in a graph. These models can be used to summarize a network of documents, predict links between them, and predict words within them.

Joint work with Jonathan Chang and Jon McAuliffe.

Bio:
David Blei is an assistant professor in the Computer Science department at Princeton University. He received his Ph.D. in 2004 from U.C. Berkeley and was a postdoctoral researcher in the Department of Machine Learning at Carnegie Mellon University. His research interests include graphical models, approximate posterior inference, and nonparametric Bayesian statistics. He focuses on applications to information retrieval and natural language processing.

Lab Meeting March 16th (fish60): Path planning in image space for autonomous robot navigation in unstructured environments

Journal of Field Robotics
Volume 26, Issue 2 (February 2009)
Special Issue on LAGR Program, Part II
Pages 212-240

Abstract:
In this paper we present an image space technique for path planning in unknown unstructured outdoor environments. Our method differs from previous techniques in that we perform path search directly in image space - the native sensor space of the imaging sensor. Our image space planning techniques can potentially be used with many different kinds of sensor data, and we experimentally evaluate the use of stereo disparity and color information. We present an extension to the basic image space planning system called the cylindrical planner that simulates a 2 field of view with a cylindrically shaped occupancy grid. We believe that image space planning is well suited for use in the local subsystem of a hierarchical planner and implement a hybrid hierarchical planner that utilizes the cylindrical planner as a local planning subsystem and a two-dimensional Cartesian planner as the global planning subsystem.
Link

Lab Meeting March 9th (slyfox): A Database and Evaluation Methodology for Optical Flow

Title: A Database and Evaluation Methodology for Optical Flow
Authors: Simon Baker, Daniel Scharstein, JP Lewis, Stefan Roth, Michael Black, Richard Szeliski
Published in: ICCV 2007

Abstract:
The quantitative evaluation of optical flow algorithms by Barron et al. led to significant advances in the performance of optical flow methods. The challenges for optical flow today go beyond the datasets and evaluation methods proposed in that paper and center on problems associated with
nonrigid motion, real sensor noise, complex natural scenes, and motion discontinuities. Our goal is to establish a new set of benchmarks and evaluation methods for the next generation of optical flow algorithms. To that end, we contribute four types of data to test different aspects of optical flow algorithms: sequences with nonrigid motion where the ground-truth flow is determined by tracking hidden fluorescent texture; realistic synthetic sequences; high frame-rate video used to study interpolation error; and modified stereo sequences of static scenes. In addition to the average angular error used in Barron et al., we compute the absolute flow endpoint error, measures for frame interpolation error, improved statistics, and flow accuracy at motion boundaries and in textureless regions. We evaluate the performance of several well-known methods on this data to establish the current state of the art. Our database is freely available on
the web together with scripts for scoring and publication of the results at http://vision.middlebury.edu/flow/

[Link]

Lab Meeting March 9th (Andi): Multilevel Surface Maps

An Efficient Extension to Elevation Maps for Outdoor Terrain Mapping and Loop Closing
Authors: Patrick Pfaff, Rudolph Triebel and Wolfram Burgard

and

Monte Carlo Localization in Outdoor Terrains Using Multilevel Surface Maps
Authors: Rainer Kuemmerle, Patrick Pfaff, Rudolph Triebel and Wolfram Burgard

Abstract: We propose a novel combination of techniques for robustly estimating the position of a mobile robot in outdoor environments using range data. Our approach applies a particle filter to estimate the full six-dimensional state of the robot and utilizes multilevel surface maps, which, in contrast to standard elevation maps, allow the robot to represent vertical structures and multiple levels in the environment. We describe probabilistic motion and sensor models to calculate the proposal distribution and to evaluate the likelihood of observations. We furthermore describe an active localization approach that actively selects the sensor orientation of the two-dimensional laser range scanner to improve the localization results. To efficiently calculate the appropriate orientation, we apply a clustering operation on the particles and evaluate potential orientations on the basis of these clusters.
Experimental results obtained with a mobile robot in large-scale outdoor environments indicate that our approach yields robust and accurate position estimates. The experiments also demonstrate that multilevel surface maps lead to a significantly better localization performance than standard elevation maps. They additionally show that further accuracy is obtained from the active sensing approach.

Links:
paper1
paper2

Lab Meeting March 2, 2009 (Casey) : Nonrigid Structure from Motion in Trajectory Space

Title: Nonrigid Structure from Motion in Trajectory Space

Authors: Ijaz Akhter, Yaser Sheikh, SohaibKhan and Takeo Kanade

Abstract:

Existing approaches to nonrigid structure from motion assume that the instantaneous 3D shape of a deforming object is a linear combination of basis shapes, which have to be estimated anew for each video sequence. In contrast, we propose that the evolving 3D structure be described by a linear combination of basis trajectories. The principal advantage of this approach is that we do not need to estimate any basis vectors during computation. We show that generic bases over trajectories, such as the Discrete Cosine Transform (DCT) basis, can be used to compactly describe most real motions. This results in a significant reduction in unknowns, and corresponding stability in estimation. We report empirical performance, quantitatively using motion capture data and qualitatively on several video sequences exhibiting nonrigid motion including piece-wise rigid motion, partially nonrigid motion(such as a facial expression), and highly nonrigid motion(such as a person dancing). 

[Link]

Lab Meeting March 2, 2009(ZhenYu): Scale Invariant Feature Matching with Wide Angle Images

Title: Scale Invariant Feature Matching with Wide Angle Images(IROS 2007)
Authors: Hansen, P. Corke, P. Boles, W. Daniilidis, K.
Abstract:

Numerous scale-invariant feature matching algorithms using scale-space analysis have been proposed for use with perspective cameras, where scale-space is defined as convolution with a Gaussian. The contribution of this work is a method suitable for use with wide angle cameras. Given an input image, we map it to the unit sphere and obtain scale-space images by convolution with the solution of the spherical diffusion equation on the sphere which we implement in the spherical Fourier domain. Using such an approach, the scale-space response of a point in space is independent of its position on the image plane for a camera subject to pure rotation. Scale invariant features are then found as local extrema in scale-space. Given this set of scale-invariant features, we then generate feature descriptors by considering a circular support region defined on the sphere whose size is selected relative to the feature scale. We compare our method to a naive implementation of SIFT where the image is treated as perspective, where our results show an improvement in matching performance.

[LINK]

Automatic Class-Specific 3D Reconstruction from a Single Image

Title: Automatic Class-Specific 3D Reconstruction from a Single Image
Authors: Han-Pang Chiu, Leslie Pack Kaelbling, Tom´as Lozano-P´erez

Abstract: Our goal is to automatically reconstruct 3D objects from a single image, by using prior 3D shape models of classes. The shape models, defined as a collection of oriented primitive shapes centered at fixed 3D positions, can be learned from a few labeled images for each class. The 3D class model can then be used to estimate the 3D shape of an object instance, including occluded parts, from a single image. We provide a quantitative valuation of the shape estimation process on real objects and demonstrate its usefulness in three applications: robot manipulation, object detection, and generating 3D 'pop-up' models from photos.

Link: pdf

CMU talk: Scaling Up Game Theory: Representation and Reasoning with Action Graph Games

Intelligence Seminar

February 24, 2009

Scaling Up Game Theory: Representation and Reasoning with Action Graph Games
Kevin Leyton-Brown
Computer Science Department, University of British Columbia

Abstract:

Game theory is the mathematical study of interaction among independent, self-interested agents. It has wide applications, including the design of government auctions (e.g., for distressed securities), urban planning, and the analysis of internet traffic patterns. Interestingly, most work in game theory is analytic; it is less common to analyze a model's properties computationally. Key reasons for this are that game representation size tends to grow exponentially in the number of players--making all but the simplest games infeasible to write down--and that even when games can be represented, existing algorithms (e.g., for finding equilibria) tend to have worst-case performance exponential in the game's size.

This talk describes Action-Graph Games (AGG), which make it possible to extend computational analysis to games that were previously far too large to consider. I will give an overview of our five-year effort developing AGGs, emphasizing the twin threads of representational compactness and computational tractability. More specifically, the first part of the talk will describe the core ideas of the AGG representation. AGGs are a fully-expressive, graph-based representation that can compactly express both strict and context-specific independencies in players' utility functions. I will illustrate the representation by describing several practical examples of games that may be compactly represented as AGGs. The second part of the talk will examine algorithmic considerations. First, I'll describe a dynamic programming algorithm for computing a player's expected utility under a given mixed-strategy profile, which is tractable for bounded-in-degree AGGs. This algorithm can be leveraged to provide an exponential speedup in the computation of best response, Nash equilibrium, and correlated equilibrium. Second, I'll describe a message-passing algorithm for computing pure-strategy Nash equilibria in symmetric AGGs, which is tractable for graphs with bounded treewidth; again, this implies an exponential speedup over the previous state of the art. Finally, I'll more briefly describe some current directions in our work on AGGs: the modeling, evaluation, and comparison of different advertising auction designs; the extension of AGGs to both temporal and stochastic settings; and the design of free software tools to make it easier for other researchers to use AGGs.

This talk is based on joint work with Albert Xin Jiang, David R.M. Thompson, and Navin A.R. Bhat.

Bio:
Kevin Leyton-Brown is an assistant professor in computer science at the University of British Columbia. He received a B.Sc. from McMaster University (1998), and an M.Sc. and PhD from Stanford University (2001; 2003). Much of his work is at the intersection of computer science and microeconomics, addressing computational problems in economic contexts and incentive issues in multi-agent systems. He also studies the application of machine learning to the design and analysis of algorithms for solving hard computational problems. He has co-written two Multiagent Essentials of Game and over forty peer-refereed technical articles. He is an associate editor of the Journal of Artificial Intelligence Research (JAIR) and a member of the editorial board of the Artificial Intelligence Journal (AIJ). He has served as a consultant for Trading Dynamics Inc., Ariba Inc., and Cariocas Inc., and is currently scientific advisor to Worio Inc.

Lab Meeting February 23, 2009(Chung-Han) Human-Assisted Motion Annotation

Title : Human-Assisted Motion Annotation

Auther : Ce Liu, William T. Freeman, Edward H. Adelson, Yair Weiss (CSAIL MIT)

Abstract :
Obtaining ground-truth motion for arbitrary, real-worldvideo sequences is a challenging but important task for both algorithm evaluation and model design. Existing ground-truth databases are either synthetic, such as the Yosemite sequence, or limited to indoor, experimental setups, such as the database developed in [5]. We propose a human-in loop methodology to create a ground-truth motion database for the videos taken with ordinary cameras in both indoor and outdoor scenes, using the fact that human beings are experts at segmenting objects and inspecting the match between two frames. We designed an interactive computer vision system to allow a user to efficiently annotate motion.Our methodology is cross-validated by showing that human annotated motion is repeatable, consistent across annotators,and close to the ground truth obtained by [5]. Using our system, we collected and annotated 10 indoor andoutdoor real-world videos to form a ground-truth motion database. The source code, annotation tool and database is online for public evaluation and benchmarking.

Presented at CVPR 2008.

[Full Text]

Lab Meeting Feburary 23, 2009 (Jimmy): Multi-robot Cooperative Localization through Collaborative Visual Object Tracking

In: RoboCup 2007

Title: Multi-robot Cooperative Localization through Collaborative Visual Object Tracking

Authors: Zhibin Liu, Mingguo Zhao, Zongying Shi, and Wenli Xu

Abstract

In this paper we present an approach for a team of robots to cooperatively improve their self localization through collaboratively tracking a moving object. At first, we use a Bayes net model to describe the multi-robot self localization and object tracking problem. Then, by exploring the independencies between different parts of the joint state space of the complex system, we show how the posterior estimation of the joint state can be factorized and the moving object can serve as a bridge for information exchange between the robots for realizing cooperative localization. Based on this, a particle filtering method for the joint state estimation problem is proposed. And, finally, in order to improve computational efficiency and achieve real-time implementation, we present a method for decoupling and distributing the joint state estimation onto different robots. The approach has been implemented on our four-legged AIBO robots and tested through different scenarios in RoboCup domain showing that the performance of localization can indeed be improved significantly.

[link]

MIT talk: Do predictions of visual perception aid design?

Do predictions of visual perception aid design?
Speaker: Ruth Rosenholtz, MIT Brain and Cognitive Sciences
Date: Friday, February 20 2009
Time: 2:00PM to 3:00PM

Understanding and exploiting the abilities of the human visual system is an important part of the design of usable user interfaces and information visualizations. Designers traditionally learn qualitative rules-of-thumb for how to enable quick, easy and veridical perception of their design. More recently, work in human and computer vision, including in our lab, has produced more quantitative models of human perception. These models often take as input arbitrary, complex images of a design. We ask whether such models aid the design process. Through a series of interactions with designers and design teams, we find that the models can help, but in somewhat unexpected ways. Based on this study, I will suggest general design principles for perceptual tools.

Ruth Rosenholtz is a Principal Research Scientist in MIT's Department of Brain and Cognitive Sciences, and a member of CSAIL. She joined MIT in 2004 after 7 years at the Palo Alto Research Center (formerly Xerox PARC). Ruth's background is in electrical engineering, particularly computer vision. More recently, however, she has studied human vision, and in particular visual search and attention. Her engineering background shows through in her focus on finding predictive mathematical models of phenomena in human vision, and in her interest in applying knowledge of human perception to the design of better user interfaces and information visualizations.

Honda Research Institute : Summer internship 2009

The computer science research section of Honda Research Institute USA, located in Mountain View, California and Cambridge, Massachusetts, has several openings for internships this summer. For details, please go to:

http://www.honda-ri.com/HRI_Us/careers/internships

The application deadline is March 31, 2009.

CMU talk: Bringing people in the loop

VASC Seminar
Monday, February 16

Bringing people in the loop: data annotation and real-time visual supervision with Amazon Mechanical Turk
Alexander Sorokin
University of Illinois at Urbana-Champaign


Abstract:

In this talk I will challenge the common assumption that "we will never have enough labeled data". As of today, large annotated datasets can be built very quickly and cheaply. To achieve that a number of questions needs to be answered. I will discuss what properties of the tasks affect the quality of the final result as well as cost and speed of task completion. I will further highlight how people in the loop framework expands the range of possible applications.

Bio:
Alexander Sorokin is a 5th year PhD student at the University of Illinois at Urbana-Champaign. He has received BS from Lomonosov Moscow State University in 2003. In 2007 he received a best paper award at InfoVis 2007 for the paper "Visualizing the History of Living Spaces".

CMU talk: Large Scale Scene Matching for Graphics and Vision

Speaker: James Hays (CSD@CMU)
Date: Monday, Feb 16, 2009
Time: 12:00 noon

Title:
Large Scale Scene Matching for Graphics and Vision

Abstract:
The complexity of the visual world makes it difficult for computer vision to understand images and for computer graphics to synthesize visual content. The traditional computer vision or computer graphics pipeline mitigates this complexity with a bottom up, divide and conquer strategy (e.g. segmenting then classifying, assembling part-based models, or using scanning window detectors). In this talk I will discuss research that is fundamentally different, enabled by the observation that while the space of images is infinite, the space of "scenes" might not be astronomically large. With access to imagery on an Internet scale, for most images there exist numerous semantically and structurally similar scenes. My research is focused on exploiting and refining large scale scene matching to short circuit the typical computer vision and graphics pipelines for tasks such as scene completion, image geolocation, object detection, and high interest photo selection.

CMU talk: Intelligent Preference Assessment: The Next Steps?

Intelligence Seminar

February 3, 2009
3:30 pm


Intelligent Preference Assessment: The Next Steps?

Craig Boutilier, Department of Computer Science, University of Toronto

Preference elicitation is generally required when making or recommending decisions on behalf of users whose utility function is not known with certainty. Full elicitation of user utility functions is infeasible in practice, leading to an emphasis on approaches that (a) attempt to make good recommendations with incomplete utility information; and (b) heuristically minimize the amount of user interaction needed to assess relevant aspects of a utility function. Current techniques are, however, limited in a number of ways: (i) they rely on specific forms of information for assessment; (ii) they require very stylized forms of interaction; (iii) they are limited in the types of decision problems that can be handled.

In this talk, I will outline several key research challenges in taking preference assessment to a point where wide user acceptance is possible. I will focus on three three current techniques we're developing that will help move in the direction of greater user acceptance. Each tackles one of the weaknesses discussed above.

1. The first two techniques allows users to define "personalized" features over which they can express their preferences. Users provide (positive and negative) instances of a concept (or feature) over which they have preferences. We relate this to models of concept learning, and discuss how they existence of utility functions allows decisions to be made with very incomplete knowledge of the target concept. I'll also discuss possible means integrating data-intensive collaborative filtering approaches with explicit preference elicitation techniques, especially when tackling "subjective" features.

2. I'll discuss some of our recent work on applying explicit decision-theoretic models to more "conversational" critiquing approaches to recommender systems. We consider several semantics (wrt user preferences) for unstructured user choices and show how these can be integrated into regret-based models.

3. Time permitting, I'll provide a sketch of some recent work on eliciting reward functions in Markov decision processes using the notion of minimax regret.

Bio:

Craig Boutilier received his Ph.D. in Computer Science (1992) from the University of Toronto, Canada. He is Professor and Chair of the Department of Computer Science at the University of Toronto. He was previously an Associate Professor at the University of British Columbia, a consulting professor at Stanford University, and a visiting professor at Brown University. He has served on the Technical Advisory Board of CombineNet, Inc. since 2001.

Dr. Boutilier's research interests span a wide range of topics, with a focus on decision making under uncertainty, including preference elicitation, mechanism design, game theory, Markov decision processes, and reinforcement learning. He is a Fellow of the American Association of Artificial Intelligence (AAAI) and the recipient of the Isaac Walton Killam Research Fellowship, an IBM Faculty Award and the Killam Teaching Award. He has also served in a variety of conference organization and editorial positions, and is Program Chair of the upcoming Twenty-first International Joint Conference on Artificial Intelligence (IJCAI-09).