Entropy-constrained tree-structured vector quantizer design by the minimum cross entropy principle

We address the variable rate tree-structured vector quantizer design problem, wherein the rate is measured by the quantizer's entropy. For this problem, tree pruning via the Generalized Breiman-Friedman-Olshen-Stone algorithm obtains solutions which are optimal over the restricted solution space consisting of all pruned trees derivable from an initial tree. We develop a joint optimization method which is inspired by the deterministic annealing algorithm for data clustering, and which extends our previous work on tree-structured vector quantization. The method is based on the principle of minimum cross entropy, using informative priors to approximate the unstructured solution while imposing the structural constraint. As in the original deterministic annealing method, the number of distinct codevectors (and hence the tree) grows by a sequence of bifurcations in the process, which occur as solutions of a free energy minimization. Our method obtains performance gains over growing and pruning methods for variable rate quantization of Gauss-Markov and Gaussian mixture sources.