CMU Intelligence Seminar: An efficient way to learn deep generative models

An efficient way to learn deep generative models

Geoff Hinton, University of Toronto

I will describe an efficient, unsupervised learning procedure for deep generative models that contain millions of parameters and many layers of hidden features. The features are learned one layer at a time without any information about the final goal of the system. After the layer-by-layer learning, a subsequent fine-tuning process can be used to significantly improve the generative or discriminative performance of the multilayer network by making very slight changes to the features.

I will demonstrate this approach to learning deep networks on a variety of tasks including: Creating generative models of handwritten digits and human motion; finding non-linear, low-dimensional representations of very large datasets; and predicting the next word in a sentence. I will also show how to create hash functions that map similar objects to similar addresses, thus allowing hash functions to be used for finding similar objects in a time that is independent of the size of the database.

Speaker Bio

Geoffrey Hinton received his BA in experimental psychology from Cambridge in 1970 and his PhD in Artificial Intelligence from Edinburgh in 1978. He did postdoctoral work at Sussex University and the University of California San Diego and spent five years as a faculty member in the Computer Science department at Carnegie-Mellon University. He then became a fellow of the Canadian Institute for Advanced Research and moved to the Department of Computer Science at the University of Toronto. He spent three years from 1998 until 2001 setting up the Gatsby Computational Neuroscience Unit at University College London and then returned to the University of Toronto where he is a University Professor. He holds a Canada Research Chair in Machine Learning. He is the director of the program on "Neural Computation and Adaptive Perception" which is funded by the Canadian Institute for Advanced Research.

Geoffrey Hinton is a fellow of the Royal Society, the Royal Society of Canada, and the Association for the Advancement of Artificial Intelligence. He is an honorary foreign member of the American Academy of Arts and Sciences, and a former president of the Cognitive Science Society. He received an honorary doctorate from the University of Edinburgh in 2001. He was awarded the first David E. Rumelhart prize (2001), the IJCAI award for research excellence (2005), the IEEE Neural Network Pioneer award (1998) and the ITAC/NSERC award for contributions to information technology (1992).