A Hybrid Metaheuristic Framework for Evolving the PROAFTN Classifier

Authors: Feras Al-Obeidat, Nabil Belacel, Juan A. Carretero, Prabhat Mahanti,

Abstract:

In this paper, a new learning algorithm based on a hybrid metaheuristic integrating Differential Evolution (DE) and Reduced Variable Neighborhood Search (RVNS) is introduced to train the classification method PROAFTN. To apply PROAFTN, values of several parameters need to be determined prior to classification. These parameters include boundaries of intervals and relative weights for each attribute. Based on these requirements, the hybrid approach, named DEPRO-RVNS, is presented in this study. In some cases, the major problem when applying DE to some classification problems was the premature convergence of some individuals to local optima. To eliminate this shortcoming and to improve the exploration and exploitation capabilities of DE, such individuals were set to iteratively re-explored using RVNS. Based on the generated results on both training and testing data, it is shown that the performance of PROAFTN is significantly improved. Furthermore, the experimental study shows that DEPRO-RVNS outperforms well-known machine learning classifiers in a variety of problems.

Keywords: Knowledge Discovery, Differential Evolution, Reduced Variable Neighborhood Search, Multiple criteria classification, PROAFTN, Supervised Learning.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1335238

References:

[1] D. Dutton and G. Conroy, "A review of machine learning," The Knowledge Engineering Review, vol. 12:4, pp. 341-367, 1996.
[2] D. Larose, Discovering Knowledge in Data: An Introduction to Data Mining. John Wiley & Sons, 2005.
[3] E. Alpaydin, "Introduction to machine learning (adaptive computation and machine learning)," MIT Press, 2004. TABLE IV DEPRO-RVNS TABLE V TABLE VI
[4] B. Roy, "Multicriteria methodology for decision aiding," Kluwer Academic, 1996.
[5] N. E. Fenton and W. Wang, "Risk and confidence analysis for fuzzy multicriteria decision making," Knowledge-Based Systems, vol. 19, no. 6, pp. 430-437, 2006.
[6] C. Zopounidis and M. Doumpos, "Multicriteria classification and sorting methods: A literature review," European Journal of Operational Research, vol. 138, no. 2, pp. 229-246, 2002.
[7] K. Jabeur and A. Guitouni, "A generalized framework for concordance/discordance-based multi-criteria classification methods," in Information Fusion, 2007 10th International Conference on, July 2007, pp. 1-8.
[8] N. Belacel, "Multicriteria assignment method PROAFTN: methodology and medical application," European Journal of Operational Research, vol. 125, no. 1, pp. 175-183, 2000.
[9] N. Belacel and M. Boulassel, "Multicriteria fuzzy assignment method: A useful tool to assist medical diagnosis," Artificial Intelligence in Medicine, vol. 21, no. 1-3, pp. 201-207, 2001.
[10] N. Belacel, P. Vincke, M. Scheiff, and M. Boulassel, "Acute leukemia diagnosis aid using multicriteria fuzzy assignment methodology," Computer Methods and Programs in Biomedicine, vol. 64, no. 2, pp. 145- 151, 2001.
[11] N. Belacel, Q. Wang, and R. Richard, "Web-integration of PROAFTN methodology for acute leukemia diagnosis," Telemedicine Journal and e-Health, vol. 11, no. 6, pp. 652-659, 2005.
[12] F. Al-Obeidat, N. Belacel, P. Mahanti, and J. A. Carretero, "Discretization techniques and genetic algorithm for learning the classification method proaftn," in Eighth International Conference On Machine Learning and Applications. Los Alamitos, CA, USA: IEEE Computer Society, 2009, pp. 685-688.
[13] R. Storn and K. Price, "Differential evolution - A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces," Journal of Global Optimization, vol. 11, no. 4, pp. 341-359, December 1997.
[14] S. Paterlini and T. Krink, "Differential evolution and particle swarm optimisation in partitional clustering," Comput. Stat. Data Anal, vol. 50, pp. 1220-1247, 2006.
[15] B. Subudhi and D. Jena, "Differential evolution and levenberg marquardt trained neural network scheme for nonlinear system identification," Neural Process. Lett., vol. 27, no. 3, pp. 285-296, 2008.
[16] M. Tayel and A. H. Yassin, "An introduced neural network-differential evolution model for small signal modeling of phemts," International Conference on Electronic Computer Technology, vol. 0, pp. 499-506, 2009.
[17] P. Hansen and N. Mladenovic, "Variable neighborhood search for the p-median," Location Science, vol. 5, pp. 207-226, 1997.
[18] P. Hansen and N. Mladenovic, "Variable neighborhood search: Principles and applications," European Journal of Operational Research, no. 130, pp. 449-467, 2001.
[19] Proceedings of the International Joint Conference on Neural Networks, IJCNN 2008, part of the IEEE World Congress on Computational Intelligence, WCCI 2008, Hong Kong, China, June 1-6, 2008. IEEE, 2008.
[20] C.-Y. Tsai and C.-C. Chiu, "A vns-based hierarchical clustering method," in CIMMACS-06: Proceedings of the 5th WSEAS International Conference on Computational Intelligence, Man-Machine Systems and Cybernetics. Stevens Point, Wisconsin, USA: World Scientific and Engineering Academy and Society (WSEAS), 2006, pp. 268-275.
[21] N. Belacel, H. Raval, and A. Punnen, "Learning multicriteria fuzzy classification method PROAFTN from data," Computers and Operations Research, vol. 34, no. 7, pp. 1885-1898, 2007.
[22] J. R. Quinlan, "Improved use of continuous attributes in c4.5," Journal of Artificial Intelligence Research, vol. 4, pp. 77-90, 1996.
[23] G. Cooper and E. Herskovits, "A bayesian method for the induction of probabilistic networks from data," Machine Learning, vol. 9, no. 4, pp. 309-347, 1992.
[24] C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 1-47, 1998.
[25] G. Castellano, A. Fanelli, and M. Pelillo, "An iterative pruning algorithm for feedforward neural networks," IEEE Transactions on Neural Networks, vol. 8, no. 3, pp. 519-531, 1997.
[26] B. Twala, "Multiple classifier application to credit risk assessment," Expert Systems with Applications, vol. In Press, Uncorrected Proof, pp. - , 2009.
[27] H. Witten, Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann Series in Data Management Systems, 2005.
[28] N. Belacel, "Multicriteria classification methods: Methodology and medical applications," Ph.D. dissertation, Free University of Brussels, Belgium, 1999.
[29] J. Kacprzyk, Advances in Differential Evolution. Springer, 2008.
[30] A. Asuncion and D. Newman, "UCI machine learning repository," 2007.
[31] S. Pang, D. Kim, and S. Bang, "Face membership authentication using SVM classification tree generated by membership-based lle data partition," IEEE Transactions on Neural Networks, vol. 16, no. 2, pp. 436-446, 2005.
[32] Y. Shirvany, M. Hayati, and R. Moradian, "Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations," Appl. Soft Comput., vol. 9, no. 1, pp. 20-29, 2009.
[33] D. Aha, "Lazy learning," Dordrecht: Kluwer Academic Publishers, 1997.
[34] D. K. Subramanian, V. S. Ananthanarayana, and M. Narasimha Murty, "Knowledge-based association rule mining using and-or taxonomies," Knowledge-Based Systems, vol. 16, no. 1, pp. 37-45, 2003.
[35] J. Demˇsar, "Statistical comparisons of classifiers over multiple data sets," Journal of Machine Learning Research., vol. 7, pp. 1-30, 2006.
[36] S. Garcia and F. Herrera, "An extension on "statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons," Journal of Machine Learning Research, vol. 9, pp. 2677-2694, 2009