Combined learning and use for classification and regression models

We show that the decision function of a radial basis function (RBF) classifier is equivalent in form to the Bayes-optimal discriminant associated with a special kind of mixture-based statistical model. The relevant mixture model is a type of `mixture of experts' model for which class labels, like continuous-valued features, are assumed to have been generated randomly, conditional on the mixture component of origin. The new interpretation shows that RBF classifiers do effectively assume a probability model which, moreover, is easily determined given the designed RBF. This interpretation also suggests a maximum likelihood learning objective, as an alternative to standard methods, for designing the RBF-equivalent models. This statistical objective is especially useful for incorporating unlabelled data within learning to enhance performance. While this approach might appear to be limited to applications involving a large, label-deficient training set, the scope of application is significantly extended with the observation that any new data to classify is also unlabelled data, available for learning. Thus, we suggest a combined learning and use paradigm, to be invoked whenever there is new data to classify. This new approach is tested for vowel recognition, given a small archive of examples from different speakers. For this problem, a conventional method is of necessity speaker-independent. By contrast, combined learning and use allows speaker-dependent adaptation, with resulting gains in performance.