Optimizing the Probabilistic Neural Network Training Algorithm for Multi-Class Identification

Authors: Abdelhadi Lotfi, Abdelkader Benyettou

Abstract:

In this work, a training algorithm for probabilistic neural networks (PNN) is presented. The algorithm addresses one of the major drawbacks of PNN, which is the size of the hidden layer in the network. By using a cross-validation training algorithm, the number of hidden neurons is shrunk to a smaller number consisting of the most representative samples of the training set. This is done without affecting the overall architecture of the network. Performance of the network is compared against performance of standard PNN for different databases from the UCI database repository. Results show an important gain in network size and performance.

Keywords: Classification, probabilistic neural networks, network optimization, pattern recognition.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.3346742

References:

[1] D. F. Specht, « Probabilistic neural networks », Neural networks, vol. 3, no. 1, p. 109–118, 1990.
[2] Y.-qun Deng and P.-ming Wang, « Predicting the shrinkage of thermal insulation mortar by probabilistic neural networks », Journal of Zhejiang University SCIENCE A, vol. 11, no. 3, p. 212-222, 2010.
[3] M. Bazarghan and R. Gupta, « Automated classification of sloan digital sky survey (SDSS) stellar spectra using artificial neural networks », Astrophysics and Space Science, vol. 315, no. 1-4, p. 201-210, 2008.
[4] C. M. Bishop, Neural networks for pattern recognition. Oxford University Press, 1995.
[5] Abdelhadi Lotfi and Abdelkader Benyettou. “Cross validation probabilistic neural network based face identification”. Journal of Information Processing Systems. Page: 1075~1086, Vol. 14, No.5, 2018. DOI: 10.3745/JIPS.04.0085.
[6] R. O. Duda, Pattern Classification 2nd Edition with Computer Manual 2nd Edition Set. John Wiley & Sons Inc, 2004.
[7] T. P. Tran, T. T. S. Nguyen, P. Tsai, and X. Kong, « BSPNN: boosted subspace probabilistic neural network for email security », Artificial Intelligence Review, 2011.
[8] T. P. Tran, L. Cao, D. Tran, and C. D. Nguyen, « Novel Intrusion Detection using Probabilistic Neural Network and Adaptive Boosting », 0911.0485, nov. 2009.
[9] M. R. Nikoo, R. Kerachian, S. Malakpour-Estalaki, S. N. Bashi-Azghadi, and M. M. Azimi-Ghadikolaee, « A probabilistic water quality index for river water quality assessment: a case study », Environmental Monitoring and Assessment, 2010.
[10] M. S. Bascil and H. Oztekin, « A Study on Hepatitis Disease Diagnosis Using Probabilistic Neural Network », Journal of Medical Systems, 2010.
[11] F. Budak and E. D. Übeyli, « Detection of Resistivity for Antibiotics by Probabilistic Neural Networks », Journal of Medical Systems, vol. 35, no. 1, p. 87-91, 2009.
[12] N. Neggaz and A. Benyettou, « Hybrid models based on biological approaches for speech recognition », Artificial Intelligence Review, vol. 32, no. 1-4, p. 45-57, 2009.
[13] M.-I. Faraj and J. Bigun, « Synergy of Lip-Motion and Acoustic Features in Biometric Speech and Speaker Recognition », IEEE Transactions on Computers, vol. 56, no. 9, p. 1169-1175, 2007.
[14] S. Meshoul and M. Batouche, « A novel approach for Online signature verification using fisher based probabilistic neural network », in Computers and Communications, IEEE Symposium on, Los Alamitos, CA, USA, 2010, vol. 0, p. 314-319.
[15] Abdelhadi Lotfi, Abdelkader Benyettou, "Over-fitting Avoidance in Probabilistic Neural Networks", World Congress of Information Technology and Computer Applications, 11-13 June 2015 Hammamet Tunisia.
[16] N. Neggaz, M. Besnassi, and A. Benyettou, « Application of Improved AAM and Probabilistic Neural network to Facial Expression Recognition », Journal of Applied Sciences, vol. 10, no. 15, p. 1572-1579, 2010.
[17] L. F. Araghi, H. Khaloozade, and M. R. Arvan, « Ship Identification Using Probabilistic Neural Networks (PNN) ».
[18] M. W. Kim et M. Arozullah, « Generalized probabilistic neural network based classifiers », in Neural Networks, 1992. IJCNN, International Joint Conference on, 2002, vol. 3, p. 648–653.
[19] V. Georgiou, P. Alevizos, and M. Vrahatis, « Fuzzy Evolutionary Probabilistic Neural Networks », Artificial Neural Networks in Pattern Recognition, p. 113–124, 2008.
[20] P. Burrascano, « Learning vector quantization for the probabilistic neural network », IEEE Transactions on Neural Networks / a Publication of the IEEE Neural Networks Council, vol. 2, no. 4, p. 458-461, 1991.
[21] I. De Falco, A. Della Cioppa, and E. Tarantino, « Facing classification problems with Particle Swarm Optimization », Applied Soft Computing, vol. 7, p. 652–658, juin. 2007.
[22] I. Galleske and J. Castellanos, « A rotated kernel probabilistic neural network (RKPNN) for multi-class classification », in Proceedings of the Artificial and natural neural networks 7th international conference on Computational methods in neural modeling - Volume 1, Berlin, Heidelberg, 2003, p. 152–157.
[23] Abdelhadi Lotfi and Abdelkader Benyettou. “A Reduced Probabilistic Neural Network for Classification of Large Databases”. Turkish Journal of Electrical Engeneering and Computer Science, 2014.
[24] Abdelhadi Lotfi and Abdelkader Benyettou. “Using Probabilistic Neural Networks for Handwritten Digit Recognition”. Journal of Artificial Intelligence, 4: 288-294. 2011.
[25] Dua, D. and Karra Taniskidou, E. (2017). UCI Machine Learning Repository (http://archive.ics.uci.edu/ml). Irvine, CA: University of California, School of Information and Computer Science.
[26] P. W. Frey and D. J. Slate. "Letter Recognition Using Holland-style Adaptive Classifiers". (Machine Learning Vol 6 #2 March 91).