Evolution of Fuzzy Neural Networks Using an Evolution Strategy with Fuzzy Genotype Values

Authors: Hidehiko Okada

Abstract:

Evolution strategy (ES) is a well-known instance of evolutionary algorithms, and there have been many studies on ES. In this paper, the author proposes an extended ES for solving fuzzy-valued optimization problems. In the proposed ES, genotype values are not real numbers but fuzzy numbers. Evolutionary processes in the ES are extended so that it can handle genotype instances with fuzzy numbers. In this study, the proposed method is experimentally applied to the evolution of neural networks with fuzzy weights and biases. Results reveal that fuzzy neural networks evolved using the proposed ES with fuzzy genotype values can model hidden target fuzzy functions even though no training data are explicitly provided. Next, the proposed method is evaluated in terms of variations in specifying fuzzy numbers as genotype values. One of the mostly adopted fuzzy numbers is a symmetric triangular one that can be specified by its lower and upper bounds (LU) or its center and width (CW). Experimental results revealed that the LU model contributed better to the fuzzy ES than the CW model, which indicates that the LU model should be adopted in future applications of the proposed method.

Keywords: Evolutionary algorithm, evolution strategy, fuzzy number, feedforward neural network, neuroevolution.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1109153

References:

[1] H. Ishibuchi, H. Tanaka, and H. Okada, Fuzzy neural networks with fuzzy weights and fuzzy biases, Proc. of IEEE International Conferences on Neural Networks, pp.1650-1655, 1993.
[2] D.B. Fogel, L.J. Fogel, and V.W. Porto, Evolving neural networks, Biological Cybernetics, vol.63, issue 6, pp.487-493, 1990.
[3] X. Yao, Evolving artificial neural networks, Proc. of the IEEE, vol.87, no.9, pp.1423-1447, 1999.
[4] K.O. Stanley and R. Miikkulainen, Evolving neural networks through augmenting topologies, Evolutionary Computation, vol.10, no.2, pp.99-127, 2002.
[5] D. Floreano, P. Durr, and C. Mattiussi, Neuroevolution: From architectures to learning, Evolutionary Intelligence, vol.1, no.1, pp.47-62, 2008.
[6] H. Okada, Genetic algorithm with fuzzy genotype values and its application to neuroevolution, International Journal of Computer, Information Science and Engineering, vol.8, no.1, pp.1-7, 2014.
[7] H-P. Schwefel, Evolution and Optimum Seeking, Wiley, 1995.
[8] H-G. Beyer and H-P. Schwefel, Evolution strategies - A comprehensive introduction, Natural Computing, vol.1, no.1, pp.3-52, 2002.
[9] N. Hansen and A. Ostermeier, Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation, Proc. of the 1996 IEEE International Conference on Evolutionary Computation, pp.312-317, 1996.
[10] M. Herdy, Evolution strategies with subjective selection, Proc. of the 4th International Conference on Parallel Problem Solving from Nature, pp.22-31, 1996.
[11] J. Knowles and D. Corne, The pareto archived evolution strategy: A new baseline algorithm for pareto multiobjective optimisation, Proc. of the 1999 Congress on Evolutionary Computation, pp.98-105, 1999.
[12] N. Hansen and A. Ostermeier, Completely derandomized self-adaptation in evolution strategies, Evolutionary Computation, vol.9, no.2, pp.159-195, 2001.
[13] H.G. Beyer, The Theory of Evolution Strategies, Springer, 2001.
[14] A. Auger and N. Hansen, A restart CMA evolution strategy with increasing population size, Proc. of the 2005 IEEE Congress on Evolutionary Computation, pp.1769-1776, 2005.
[15] O.M. Shir and T. Back, Niching in evolution strategies, Proc. of the 2005 Conference on Genetic and Evolutionary Computation, pp.915-916, 2005.
[16] N. Hansen, The CMA evolution strategy: A comparing review, Towards a New Evolutionary Computation, Springer, pp.1769-1776, 2006.
[17] D.V. Arnold, Weighted multirecombination evolution strategies, Theoretical Computer Science - Foundations of Genetic Algorithms, vol.361, no. 1, pp.18-37, 2006.
[18] X. Chen, X. Liu, and Y. Jia, Combining evolution strategy and gradient descent method for discriminative learning of bayesian classifiers, Proc. of the 11th Annual Conference on Genetic and Evolutionary Computation, pp.507-514, 2009.
[19] D.V. Arnold and A.S. Castellarin, A novel approach to adaptive isolation in evolution strategies, Proc. of the 11th Annual Conference on Genetic and Evolutionary Computation, pp.491-498, 2009.
[20] L. Graening, N. Aulig, and M. Olhofer, Towards directed open-ended search by a novelty guided evolution strategy, Proc. of the 11th International Conference on Parallel Problem Solving from Nature: Part II, pp.71-80, 2010.
[21] A. Auger, D. Brockhoff, and N. Hansen, Analyzing the impact of mirrored sampling and sequential selection in elitist evolution strategies, Proc. of the 11th Workshop on Foundations of Genetic Algorithms, pp.127-138, 2011.
[22] A. Auger, D. Brockhoff, and N. Hansen, Mirrored sampling in evolution strategies with weighted recombination, Proc. of the 13th Annual Conference on Genetic and Evolutionary Computation, pp.861-868, 2011.
[23] I. Loshchilov, M. Schoenauer, and M. Sebag, Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy, Proc. of the Fourteenth International Conference on Genetic and Evolutionary Computation Conference, pp.321-328, 2012.
[24] J. Lu, B. Li, and Y. Jin, An evolution strategy assisted by an ensemble of local gaussian process models. Proc. of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, pp.447-454, 2013.
[25] R. Li, M.T.M. Emmerich, J. Eggermont, T. Back, M. Schutz, J. Dijkstra, and J.H.C. Reiber, Mixed integer evolution strategies for parameter optimization, Evolutionary Computation, vol.21, no.1 pp.29-64, 2013.
[26] L.A. Zadeh, The concept of a linguistic variable and its application to approximate reasoning - I, II, and III, Information Sciences, vol.8, pp.199-249, pp.301-357, and vol.9, pp.43-80, 1975.
[27] G. Alefeld and J. Herzberger, Introduction to Interval Computation, Academic Press, 1983.
[28] H. Okada, J. Tokida, and Y. Fujii, Comparison of evolution strategy, genetic algorithm and their hybrids on evolving autonomous game controller agents, International Journal of Science and Engineering Investigations, 10612-02, vol.1, no. 6, pp.11-16, 2012.