Integrated ACOR/IACOMV-R-SVM Algorithm

Authors: Hiba Basim Alwan, Ku Ruhana Ku-Mahamud

Abstract:

A direction for ACO is to optimize continuous and mixed (discrete and continuous) variables in solving problems with various types of data. Support Vector Machine (SVM), which originates from the statistical approach, is a present day classification technique. The main problems of SVM are selecting feature subset and tuning the parameters. Discretizing the continuous value of the parameters is the most common approach in tuning SVM parameters. This process will result in loss of information which affects the classification accuracy. This paper presents two algorithms that can simultaneously tune SVM parameters and select the feature subset. The first algorithm, ACO_R-SVM, will tune SVM parameters, while the second IACO_MV-R-SVM algorithm will simultaneously tune SVM parameters and select the feature subset. Three benchmark UCI datasets were used in the experiments to validate the performance of the proposed algorithms. The results show that the proposed algorithms have good performances as compared to other approaches.

Keywords: Continuous ant colony optimization, incremental continuous ant colony, simultaneous optimization, support vector machine.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1314883

References:

[1] Chen, W., Chun-Na, L., Hua-Xin, P., Yan-Ru, G., & Yuan-Hai, S. (2017). Alternating direction method of multipliers for L1- and L2-norm best fitting hyper plane classifier. Procedia Computer Science, 108C (2017), 1292-1301.
[2] Kojo, G, James, B., Andrew H., & Elena, G. (2017). Linear classifier design under heteroscedasticity in linear discriminant analysis. Expert System with Applications, 79 (15 Augest 2017), 4-52.
[3] Acharya, J., Das, A., Orlitsky, A., Pan, S. & Santhanam, N. (2010). Classification using pattern probability estimators. Proceeding of The International Symposium on Information theory, 13-18 June 2010, Austin, TX, 1493 - 1497.
[4] L., Demidove, I., Klyueva, Y., Sokolova, N., Stepanov, & N., Tyart. (2017). Intellectual approaches to improvement of the classification decisions quality on the base of the SVM classifier. Procedia Computer Science, 103 (2017), 222-230.
[5] Craig, V., Stephen, V., Conrad, J., James, A., & Gregory, H. (2015). MapReduce SVM game. Procedia Computer Science, 53 (2015), 298-307.
[6] Wang, T., Huang H., Tian S. & Xu J. (2010). Feature selection for SVM via optimization of kernel polarization with gaussian ARD kernels. Expert Systems with Applications, 37(9), 6663-6668.
[7] Basiri, M., Aghaee, N. & Aghdam, M. (2008). Using ant colony optimization-based selected features for predicting post-synaptic activity in proteins. In E. Marchiori & J. Moore (Eds.), Evolutionary computation, machine learning and data mining in bioinformatics, (pp. 12-23). Berlin Heidelberg: Springer.
[8] Qiu, D., Li, Y., Zhang, X. & Gu, B. (2011). Support vector machine with parameter optimization by bare bones differential evolution. Proceedings of The 7th International Conference of the Natural Computation, 26-28 July 2011, Shanghai, 263-266.
[9] Wen, S., Yao, M., Wu, C., & Li, J. (2017). An improved fastSLAM2.0 algorithm based on ant colony optimization. Proceedings of The 29th Conference of Control and Decision, 28-30 May 2017, Chongqing, China, 1948-9447.
[10] Socha, K. (2008). Ant colony optimization for continuous and mixed-variables domain. (Doctoral dissertation, Universite’ Libre de Bruxelles, 2008). Retrieved from iridia.ulb.ac.be/~mdorigo/HomePageDorigo/thesis/SochaPhD.pdf.
[11] Socha, K. (2004). ACO for continuous and mixed-variable optimization. In M. Dorigo, M. Birattari, C. Blum, L. Gambardella & F. Mondada (Eds.), Ant Colony Optimization and Swarm Intelligence, (pp. 25-36). Berlin Heidelberg: Springer.
[12] Socha, K. & Dorigo, M. (2008). Ant colony optimization for continuous domain. European Journal of Operational Research, 185(3), 1155-1173.
[13] Liao, T. (2011). Improved ant colony optimization algorithms for continuous and mixed discrete-continuous optimization problems (Report No. 2010/2011). Retrieved from IRIDIA, Universite Libre de Bruxelles: Brussels, Belgium website: http://iridia.ulb.ac.be/~tliao/papers/DEAreport-Liao.pdf.
[14] Liao, T., Oca, M., Aydin, D., Stützle, T. & Dorigo, M. (2011). An incremental ant colony algorithm with local search for continuous optimization (Report No. TR/IRIDIA/2011-005). Retrieved from IRIDIA, Universite Libre de Bruxelles: Brussels, Belgium website: http://iridia.ulb.ac.be/IridiaTrSeries/rev/IridiaTr2011-005r002.pdf.
[15] Alwan, H.B. and Ku-Mahamud, K.R. (2017). Mixed-variable ant colony optimization algorithm for feature subset selection and tuning SVM parameter. International Journal of Bio-Inspired Computational. 9(1), 53-63.
[16] Alwan, H. B. & Ku-Mahamud, K. R. (2013). Feature selection and model selection algorithm using incremental mixed variable ant colony optimization for support vector machine classifier. International Journal of Mathematics and Computers in Simulation. 7(5), 406-414.
[17] UCI Repository of Machine Learning Databases, Department of Information and Computer Science, University of California, Irvine, CA, , 2012.
[18] Huang, C. (2009). ACO-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing, 73(1-3), 438-448.