Evolving Neural Networks using Moment Method for Handwritten Digit Recognition

Authors: H. El Fadili, K. Zenkouar, H. Qjidaa

Abstract:

This paper proposes a neural network weights and topology optimization using genetic evolution and the backpropagation training algorithm. The proposed crossover and mutation operators aims to adapt the networks architectures and weights during the evolution process. Through a specific inheritance procedure, the weights are transmitted from the parents to their offsprings, which allows re-exploitation of the already trained networks and hence the acceleration of the global convergence of the algorithm. In the preprocessing phase, a new feature extraction method is proposed based on Legendre moments with the Maximum entropy principle MEP as a selection criterion. This allows a global search space reduction in the design of the networks. The proposed method has been applied and tested on the well known MNIST database of handwritten digits.

Keywords: Genetic algorithm, Legendre Moments, MEP, Neural Network.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1082059

References:

[1] M. K. Hu, "Visual pattern recognition by moment invariants," IRE Transaction on Information Theory, vol. 8, no. 2, pp. 179-187, 1962.
[2] S. X. Liao and Miroslaw Pawlak, "On image analysis by moments," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 18, no. 3, pp. 254-266, 1996.
[3] H. Qjidaa and L. Redouane, "Robust line fitting in a noisy image by the method of moments," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 21, no. 11, pp. 1216-1223, 1999.
[4] W. H. schiffmann and K. Mecklenburg, "Genetic Generation of Backpropagation Trained Neural Networks," Proc. of Parallel Processing in Neural Systems and Computers(ICNC), Eckmiller R. et al. (Eds.) pp. 205-208, Elsevier, 1990.
[5] G. Miller P. M. Todd and S. U. Hegde, Designing Neural Networks using Genetic Algorithms, Proc. Of the third Intern. Conference on Genetic Algorithms (ICGA), San Mateo (CA), 1989, pp. 379-384.
[6] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no.11, pp. 2278-2324, November 1998.
[7] H. El Fadili, K. Zenkouar and H. Qjidaa, "Lapped Block Image Analysis Via the Method of Legendre Moments," EURASIP Journal on Applied Signal Processing, vol. 2003, no.9, pp. 902-913, August 2003.
[8] X. Zhunang, R. M. Haralick, and Y. Zhao, "Maximum entropy image reconstruction," IEEE Trans. Signal Processing, vol. 39, no. 6, pp. 1478-1480, 1991.
[9] D. Parisi, A.Cangelosi and S. Nolfi, "cell division and migration in a genotype for neural networks," Network: computation in neural systems, vol. 5, no. 4, 1994.
[10] D. Whitley, T. Starkweather, and C. Bogart, "Genetic algorithms and neural networks: optimizing connections and connectivity," Parallel Computing,vol. 14, pp. 347-361, 1990.
[11] Y. Liu and X. Yao (1996), "A population-based learning algorithm which learns both architectures and weights of neural networks," Chinese Journal of Advanced Software Research (Allerton Press, Inc., New York, NY 10011), vol. 3, no. 1, pp. 54-65, 1996.