A Hybrid Feature Subset Selection Approach based on SVM and Binary ACO. Application to Industrial Diagnosis

Authors: O. Kadri, M. D. Mouss, L.H. Mouss, F. Merah

Abstract:

This paper proposes a novel hybrid algorithm for feature selection based on a binary ant colony and SVM. The final subset selection is attained through the elimination of the features that produce noise or, are strictly correlated with other already selected features. Our algorithm can improve classification accuracy with a small and appropriate feature subset. Proposed algorithm is easily implemented and because of use of a simple filter in that, its computational complexity is very low. The performance of the proposed algorithm is evaluated through a real Rotary Cement kiln dataset. The results show that our algorithm outperforms existing algorithms.

Keywords: Binary Ant Colony algorithm, Support VectorMachine, feature selection, classification.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1083031

References:

[1] A. L. Blum and P. Langley, "Selection of relevant features and examples in machine learning artificial," Artificial Intelligence, vol. 97, pp. 245-271, 1997.
[2] P.M. Narendra and K. Fukunaga, "A branh and bound algorithm for feature subset selection." IEEE Transactions on Computers, vol 26, pp. 917-922, 1977.
[3] J. Kittler, "Feature set search algorithms," In C. H. Chen, editor, Pattern Recognition and Signal Processing. Sijhoff and Noordhoff, the Netherlands, 1978.
[4] J. H. Yang and V. Honavar, "Feature subset selection using a genetic algorithm," IEEE Intelligent Systems, Vol. 13, no. 2, pp. 44-49, 1998.
[5] V. Maniezzo, M. Dorigo and A. Colorni, "The ant system: Optimization by a colony of cooperating agents," IEEE Transactions on Systems, Man, and Cybernetics-Part B, vol 26, no.1, pp. 29-41, 1996.
[6] X.Weiqing, Y.Chenyang and W.Liuyi, "Binary Ant Colony Evolutionary Algorithm," International Journal of Information Technology, Vol. 12, no. 3, 2006.
[7] R.Mandaoui, L.H. Mouss, "NEFDIAG. a New Approach for Industrial diagnosis by Neuro-Fuzzy systems: Application to manufacturing System," ERIMA, vol.2, no. 2, pp. 144-151, 2008.
[8] M.L. Raymer, W.F. Punch, E.D. Goodman, L.A. Kuhn and A.K Jain, "Dimensionality Reduction Using Genetic Algorithms," IEEE Transactions on Evolutionary Computation, vol 4, no. 2, pp 164-171, 2000.
[9] A. Al-Ani, "Feature Subset Selection Using Ant Colony Optimization," International Journal of Computational Intelligence, vol 2, no. 1, pp 53¬58, 2005.
[10] Z. Zhang, W. Cheng and X. Zhou, "Research on Intelligent Diagnosis of Mechanical Fault Based on Ant Colony Algorithm," The Sixth International Symposium on Neural Networks, Springer Berlin, Heidelberg, Vol 56, pp 631-640, 2009.
[11] E. Youn, "Support vector-based feature selection using Fisher's linear discriminant and Support Vector Machine," Expert Systems with Applications, vol, 37, no 9, pp. 6148-6156, 2010.
[12] Q. Wu, Fault diagnosis model based on Gaussian support vector classifier machine. Expert Systems with Applications, vol, 37, no 9, pp.6251-6256, 2010.
[13] I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," Journal of Machine Learning Research, vol 3, pp 1157— 1182, 2003.
[14] M. Dash and H. Liu, "Feature selection methods for classifications," Intelligent Data Analysis, vol 3, no.1, pp. 131-156, 1997.
[15] N. Wyse, R. Dubes, and A.K. Jain, "A critical evaluation of intrinsic dimensionality algorithms," In E.S. Gelsema and Kanal L.N, editors, Pattern Recognition in Practice, Morgan Kaufmann Publishers, Inc, pp 415-425. 1980.
[16] M. Dorigo and G. Di Caro, "The ant colony optimization meta-heuristic," In New Ideas in Optimization, McGraw-Hill, London, UK, pp 11-32, 1999.
[17] R. 0. Duda, P. E.Hart and D. G. Stork, in Pattern classification, 2nd ed. Wiley Interscience Publication, 2001.
[18] A. Asuncion, and D. J. Newman, UCI machine learning repository, 2007.
[19] S. M. Vieira, J.C. Sousa and T.A. Runkler, "Two cooperative ant colonies for feature selection using fuzzy models," Expert Systems with Applications, vol. 37, no.4, pp. 2714-2723, 2010.
[20] C. L. Huang, "ACO-based hybrid classification system with feature subset selection and model parameters optimization," Neurocomputing, vol 73, pp 438-448, 2009.
[21] M. F. Pasha, R.Budtarto, "Evolvable-NEURAL- Based Fuzzy Inference System and Its Application for Adaptive Network Anomaly Detection,"Advances in machine learning and cybernetics, vol. 3930, pp. 662-671, 2006.
[22] 0. Boz, "Feature subset selection by using sorted feature relevance," International Conference on Machine Learning and Application, Las Vegas City, P147-153, 2002.
[23] W.N.Vapnik, "An overview of statistical learning theory," IEEE Transactions of Neural Networks, vol 10, pp. 988-999, 1999.