Judges System for Classifiers Specialization

Authors: Abdel Rodríguez, Isis Bonet, Ricardo Grau, María M. García

Abstract:

In this paper we designed and implemented a new ensemble of classifiers based on a sequence of classifiers which were specialized in regions of the training dataset where errors of its trained homologous are concentrated. In order to separate this regions, and to determine the aptitude of each classifier to properly respond to a new case, it was used another set of classifiers built hierarchically. We explored a selection based variant to combine the base classifiers. We validated this model with different base classifiers using 37 training datasets. It was carried out a statistical comparison of these models with the well known Bagging and Boosting, obtaining significantly superior results with the hierarchical ensemble using Multilayer Perceptron as base classifier. Therefore, we demonstrated the efficacy of the proposed ensemble, as well as its applicability to general problems.

Keywords: classifiers, delegation, ensemble

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1061563

References:

[1] L. Breiman, Bagging predictors. Machine Learning, 1996. 24: p. 123- 140.
[2] Y. Freund and R. E. Schapire, Experiments with a new boosting algorithm. Thirteenth International Conference on Machine Learning, 1996: p. 148-156.
[3] R. E. Schapire, The strength of weak learnability. Machine Learning, 1990. 5(2): p. 197-227.
[4] R. A. Jacobs, S. J. Nowlan, and G. E. Hinton, Adaptative mixtures of local experts. Neural Computation, 1991. 3: p. 79-87.
[5] M. J. Jordan and R. A. Jacobs, Hirarchical mixtures of experts and the EM algorithm. Neural Computation, 1994. 6: p. 79-87.
[6] D. Wolpert, Stacked generalization. Neural Networks, 1992. 5(2): p. 241-259.
[7] D. J. N. A. Asuncion. UCI Machine Learning Repository. 2007.
[8] C. Ferri, P. Flach, and J. Hernández-Orallo. Delagating Classifiers. in 21st International Conference on Machine Learning. 2004. Canada.