Combining Diverse Neural Classifiers for Complex Problem Solving: An ECOC Approach

Authors: R. Ebrahimpour, M. Abbasnezhad Arabi, H. Babamiri Moghaddam

Abstract:

Combining classifiers is a useful method for solving complex problems in machine learning. The ECOC (Error Correcting Output Codes) method has been widely used for designing combining classifiers with an emphasis on the diversity of classifiers. In this paper, in contrast to the standard ECOC approach in which individual classifiers are chosen homogeneously, classifiers are selected according to the complexity of the corresponding binary problem. We use SATIMAGE database (containing 6 classes) for our experiments. The recognition error rate in our proposed method is %10.37 which indicates a considerable improvement in comparison with the conventional ECOC and stack generalization methods.

Keywords: Error correcting output code, combining classifiers, neural networks.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1079272

References:

[1] S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice- Hall, Englewood Cliffs, NJ, 1999.
[2] R.Ghaderi, Arranging simple neural network to solve complex classification problem, PhD thesis, Surrey University, 2000.
[3] Dietterich, T.G., G. Bakiri, "Error-correcting output codes: A general method for improving multiclass inductive learning programs", Proc. of the 9th National Conference on Artificial Intelligence, AAAI-91. AAAI Press, pp. 572-577, 1991.
[4] T. Windeatt, R. Ghaderi, "Binary labeling and decision level fusion", Inform. Fusion 2, 103-112, 2001.
[5] T. Windeatt, R. Ghaderi, "Coding and decoding strategies for multi-class learning problems", Inform. Fusion 4, 11-21, 2003.
[6] J. Kittler, S. A. Hojjatoleslami, T. Windeatt." Weighting Factors in Multiple Expert Fusion", In Proc. Of British Machine vision Conference BMVC 97, pp. 42-50, Essex University, Essex U.K, 1997.
[7] T.Windeatt , R. Ghaderi, "Dynamic Weighting Factors for Decision Combining", Proc. of IEE Int. Conf. On Data Fusion, Great Malvern, UK, pp. 123-130, October 1998.
[8] N. Hatami, R. Ebrahimpour, " Combining Multiple Classifiers: Diversify with Boosting and Combining by Stacking" International Journal of Computer Science and Network Security (IJCSNS), Vol.7, No.1, pp. 127-131, 2007.
[9] N. Hatami, R. Ebrahimpour, R. Ghaderi, " Error Correcting Output Codes with Weighted Minimum Distance Decoding by Genetic Algorithm" Proc. of the 3th Information and Knowledge Technology (IKT07), Ferdowsi University of Mashad, Mashad, Iran, Nov. 27-29, 2007.
[10] J.Kitter , R.Ghaderi ,T.Windeatt , J.mtas "Face verification via Error correcting output codes " International Journal of Image and Vision Computing Vol.21,pp.1163-1169,2003.
[11] T. Windeatt and R. Ghaderi. Binary codes for multi-class decision combining. Volume 4051, pages 23-34, Florida,USA, April 2000. 14th Annual International Conferenceof Society of Photo-Optical Instrumentation Engineers (SPIE).
[12] R. Ghaderi, T. Windeatt, Least squares and estimation measures via error correcting output code, in: 2nd International Workshop on Multiple Classifier Systems, Lecture Notes in Computer Science, Springer-Verlag, July 2001, pp. 148-157.
[13] T.G. Dietterich, Ensemble methods in machine learning, in: J. Kittler, F. Roli (Eds.), Multiple Classifier Systems, MCS2000,Cagliari, Italy, Springer Lecture Notes in Computer Science, 2000,pp. 1-15.
[14] T.G Dietterich, G. Bakiri, Error-correcting output codes: A general method for improving multiclass inductive learning programs, in: Proceedings of the Ninth National Conference on Artificial Intelligence (AAAI-91), AAAI Press, 1991, pp. 572-577.
[15] T.G. Dietterich, G. Bakiri, Solving multi-class learning problems via error-correcting output codes, Journal of Artificial Intelligence Research 2 (1995) 263-286.
[16] E.B. Kong, T.G. Diettrich, Error-correcting output coding corrects bias and variance, in: M. Kaufmann (Ed.), 12th Interna-20 T. Windeatt, R. Ghaderi / Information Fusion 4 (2003) 11-21tional Conference of Machine Learning, San Fransisco, 1995, pp.313-321.