CMU talk:Material Recognition By Humans and Machines

VASC Seminar Series

Material Recognition By Humans and Machines
Lavanya Sharan
MIT
2:00pm, Friday, March 20th


Abstract:
We can easily tell if a spoon is made of stainless steel or plastic, if a shirt is clean or if food is fresh. These judgments of material appearance are ubiquitous. We use our material perception abilities to decide where to step on an icy sidewalk, which items to pick in the fresh produce aisle, and if a rash requires a trip to the doctor. In spite of the importance of these judgments, little is known about material recognition in the fields of human vision or computer vision.

We have studied human material judgments on real world photographs by asking observers questions like "Is that object made of paper or plastic?" or "Are those flowers fake or real?". We find that observers can recognize materials very well, even when images are presented very fast (40 millisecond/image). This performance was robust to low-level image degradations like removal of color, blurring and inversion of contrast polarity, suggesting that low-level information is not crucial for observers.

What do these results imply for machine vision systems? We evaluated the performance of simple classifiers based on low-level image features (e.g. jet-like features, SIFT) at the same material categorization task that humans did. We find that low-level features are not sufficient for categorizing materials on our data set suggesting a parallel with the results from human experiments. We conclude that there is rich territory to be explored both by computer vision and human vision researchers for this problem.

Bio:
Lavanya Sharan is a final year graduate student working with Ted Adelson at MIT. Her research interests lie at the intersection of human vision and computer vision, especially in the domain of material recognition. She is interested in understanding how humans can recognize the materials that objects are made of and how to make computer vision systems that can do the same. Lavanya received her M.S. degree in Computer Science from MIT in 2005 and her undergraduate training in Electrical Engineering from IIT Delhi in 2003.