Information-theoretic framework for optimization with application to supervised learning

A unified approach is developed for hard optimization problems involving data association, i.e. the assignment of elements viewed as 'data' to one of a set classes so as to minimize the resulting cost. The diverse problems which fit this description include data clustering, statistical classifier design to minimize probability of error, piece-wise regression structure vector quantization, as well as optimization in graph theory. Whereas standard descent-based methods are susceptible to finding poor local optima of the cost, the suggested approach provides some potential for avoiding local optima, yet without the computational complexity of stochastic annealing.