A New Self-Adaptive EP Approach for ANN Weights Training

Authors: Kristina Davoian, Wolfram-M. Lippe

Abstract:

Evolutionary Programming (EP) represents a methodology of Evolutionary Algorithms (EA) in which mutation is considered as a main reproduction operator. This paper presents a novel EP approach for Artificial Neural Networks (ANN) learning. The proposed strategy consists of two components: the self-adaptive, which contains phenotype information and the dynamic, which is described by genotype. Self-adaptation is achieved by the addition of a value, called the network weight, which depends on a total number of hidden layers and an average number of neurons in hidden layers. The dynamic component changes its value depending on the fitness of a chromosome, exposed to mutation. Thus, the mutation step size is controlled by two components, encapsulated in the algorithm, which adjust it according to the characteristics of a predefined ANN architecture and the fitness of a particular chromosome. The comparative analysis of the proposed approach and the classical EP (Gaussian mutation) showed, that that the significant acceleration of the evolution process is achieved by using both phenotype and genotype information in the mutation strategy.

Keywords: Artificial Neural Networks (ANN), Learning Theory, Evolutionary Programming (EP), Mutation, Self-Adaptation.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1329396

References:

[1] X. Yao, "Evolutionary artificial neural networks", in Encyclopedia of Computer Science and Technology, Vol. 33, New York: Marcel Dekker, pp. 137-170, 1995
[2] X. Yao, "Evolving Artificial Neural Networks", in Proc. of the IEEE, 87 (9), pp. 1423-1447, 1999
[3] D. B. Fogel, "Evolving Neural Networks: Selected Medical Applications and the Effects of Variation Operators", Modeling and Simulation: Theory and Practice - A Memorial Volume for Professor Walter J. Karplus, Kluwer Academic Press, Boston, MA, pp. 217-248, 2003
[4] D. G. Landavazo and G. B. Fogel, "Evolved Neural Networks for Quantitative Structure-Activity Relationships of Anti-HIV Compounds", in Proc. of the IEEE Congress on Evolutionary Computation, Vol. 1, Honolulu, HI, USA, pp. 199-204, 2002
[5] A. Abraham, "Meta-Learning Evolutionary Artificial Neural Networks", Neurocomputing Journal, Elsevier Science, Netherlands, Vol. 56c, pp. 1-38, 2004
[6] A. E. Eiben, R. Hinterding, Z. Michalewicz, "Parameter Control in Evolutionary Algorithms", IEEE Trans. on Evolutionary Computation, Vol. 3, pp. 124-141, 2000
[7] R. Hinterding, "Gaussian mutation and self-adaption for numeric genetic algorithms", in Proc. of the Second IEEE Conference on Evolutionary Computation, pp. 384-389, 1995
[8] A. Jain, D. Fogel. "Case studies in applying fitness distributions in evolutionary algorithms: I. Simple neural networks and Gaussian mutation", Applications and Science of Computational Intelligence III, Proc. SPIE, Vol. 4055, pp. 168-175, 2000
[9] P. A. Castillo, J. J. Merelo, V. Rivas, G. Romero, and A. Prieto, "Evolving Multilayer Perceptrons", Neural Processing Letters 12(2), pp.115-127, 2000
[10] J. Branke, "Evolutionary approaches to dynamic optimization problems - a survey", GECCO Workshop on Evolutionary Algorithms for Dynamic Optimization Problems, pp. 134-137, 1999
[11] D. B. Fogel and K. Chellapilla, "Revisiting evolutionary programming", in SPIE AeroSense'98, Applications and Science of Computational Intelligence, Orlando, FL, pp. 2-11, 1998
[12] W.-M. Lippe, "Soft-Computing mit Neuronalen Netzen, Fuzzy-Logic und Evolutionären Algorithmen", Springer-Verlag, Berlin Heidelberg, 2006
[13] H. Abbass and R. Sarker, "Simultaneous evolution of architectures and connection weights in anns", in Artificial Neural Networks and Expert Systems Conference, Dunedin, New Zealand, pp. 16-21, 2001
[14] Lock and C. Giraud-Carrier. "Evolutionary Programming of Near- Optimal Neural Networks", in Proc. of the Fourth International Conference on Artificial Neural Networks and Genetic Algorithms (ICANNGA99), Springer-Verlag, pp. 302-306, 1999
[15] X. Yao and Y. Liu, "Fast Evolutionary Programming", in Proc. of the Fifth Annual Conference on Evolutionary Programming (EP'96), the MIT Press, San Diego, CA, USA, 29/2-2/3/96. pp. 451-460, 1996
[16] X. Yao and Y. Liu, "Fast evolution strategies," Control and Cybernetics, 26(3), pp. 467-496, 1997
[17] X. Yao, Y. Liu, and G. Lin, "Evolutionary programming made faster", IEEE Transactions on Evolutionary Computation, pp. 82-102, 1999
[18] K. Davoian, A.Reichel, W.-M. Lippe, "Comparison and analysis of mutation-based evolutionary algorithms for ANN parameters optimization", in Proc. of the 2006 International conference on Data Mining (DMIN-06),CSREA Press, 2006
[19] X. Yao, Y. Liu, "A new evolutionary system for evolving artificial neural networks", IEEE Transactions on Neural Networks, 8(3): 694- 713, May 1997
[20] Y. Chen, B. Yang, J. Dong, A. Abraham, "Time series forecasting using flexible neural tree model", Information Sciences: an International Journal, Vol 174, pp. 219-235, 2005
[21] W. Greiner, L. Neise, H. Stöcker, "Thermodynamics and statistical mechanics", Springer-Verlag, New York
[u.a.] 2000
[22] I. Rechenberg, "Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution", Fromman-Holzboog Verlag, Stuttgart, Germany, 1973
[23] M. Mackey and L. Glass, "Oscillation and chaos in physiological control systems", Sci., vol. 197, p. 287, 1977