Lab Meeting December 18th, 2007 (Atwood): Maximum Entropy Model and Conditional Random Field

I will talk about the relation between Maximum Entropy Model and Conditional Random Field, and my recent experiments.

Abstract:
In this chapter, we will describe a statistical model that conforms to the
maximum entropy principle (we will call it the maximum entropy model, or
ME model in short) [68, 69]. Through mathematical derivations, we will show
that the maximum entropy model is a kind of exponential model, and is a close
sibling of the Gibbs distribution described in Chap. 6. An essential difference
between the two models is that the former is a discriminative model, while
the latter is a generative model. Through a model complexity analysis, we will
show why discriminative models are generally superior to generative models in
terms of data modeling power. We will also describe the Conditional Random
Field (CRF), one of the latest discriminative models in the literature, and
prove that CRF is equivalent to the maximum entropy model.

Fulltext