Constraint-based, transductive learning for distributed ensemble classification

We consider ensemble classification when there is no common labeled data for designing the function which aggregates classifier decisions. In recent work, we dubbed this problem distributed ensemble classification, addressing e.g. when local classifiers are trained on different (e.g. proprietary, legacy) databases or operate on different sensing modalities. Typically, fixed (untrained) rules of classifier combination such as voting methods are used in this case. However, these may perform poorly, especially when the local class priors, used in training, differ from the true (test batch) priors. Alternatively, we proposed a transductive strategy, optimizing the combining rule for an objective function measured on the test batch. We proposed both maximum likelihood (ML) and information-theoretic (IT) objectives and found that IT achieved superior performance. Here, we identify that the fundamental advantage of the IT method is its ability to properly account for statistical redundancy in the ensemble. We also develop an extension of IT that improves its performance. Experiments are conducted on the UC Irvine machine learning repository.