Approximate maximum entropy learning for classification: Comparison with other methods

Lian Yan, David J. Miller

Research output: Contribution to conferencePaperpeer-review

1 Scopus citations


Recently, we proposed new methods for approximately learning the maximum entropy (ME) joint pmf for discrete feature spaces. Our approximate techniques overcome the intractability that plagues most ME learning methods when given a general set of constraints. The resulting models are useful for classification as well as more general inference. Our method has been demonstrated to yield strong performance in comparison with Bayesian networks, dependence trees, tree-augmented naive Bayes models, and multilayer perceptrons. After first reviewing our method, we provide insight into why it works and how it is related to, albeit distinct from naive Bayes classification (NBC). The connection to NBC then naturally leads us to suggest a simple method for parsimoniously choosing the set of constraints to encode when forming the model. Finally, we provide new experimental comparisons for our method, with decision trees and with support vector machines.

Original languageEnglish (US)
Number of pages10
StatePublished - 2001

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Software
  • Electrical and Electronic Engineering


Dive into the research topics of 'Approximate maximum entropy learning for classification: Comparison with other methods'. Together they form a unique fingerprint.

Cite this