Paper: Markov Random Fields

Hinton, G. E., Osindero, S. and Bao, K.
Learning Causally Linked Markov Random Fields.
In: Artificial Intelligence and Statistics, 2005, Barbados

Abstract:We describe a learning procedure for a generative model that contains a hidden Markov Random Field (MRF) which has directed connections to the observable variables. The learning procedure uses a variational approximation for the posterior distribution over the hidden variables. Despite the intractable partition function of the MRF, the weights on the directed connections and the variational approximation itself can be learned by maximizing a lower bound on the log probability of the observed data. The parameters of the MRF are learned by using the mean field version of contrastive divergence [1]. We show that this hybrid model simultaneously learns parts of objects and their inter-relationships from intensity images. We discuss the extension to multiple MRF's linked into in a chain graph by directed connections.