Deterministic annealing for trellis quantizer and HMM design using Baum-Welch re-estimation

The deterministic annealing algorithm for data clustering is extended to address the trellis quantizer design problem. The approach is derived within information theory and probability theory, using the principle of maximum entropy to induce a distribution over all possible path encodings of the training set. The resulting method is intimately connected to estimation procedures on Markov chains. Performance gains over known methods are obtained for memoryless, multimodal scalar sources as well as for the vector Gaussian and Laplacian sources. The method is also suggested for an estimation problem in hidden Markov models. For a Gaussian mixture state example, this approach achieves a greater likelihood value than the best result of standard Baum-Welch re-estimation, based on numerous initializations within the data.