Self-evolving Neural Networks Based On PSO and JPSO Algorithms

Authors: Abdussamad Ismail, Dong-Sheng Jeng

Abstract:

A self-evolution algorithm for optimizing neural networks using a combination of PSO and JPSO is proposed. The algorithm optimizes both the network topology and parameters simultaneously with the aim of achieving desired accuracy with less complicated networks. The performance of the proposed approach is compared with conventional back-propagation networks using several synthetic functions, with better results in the case of the former. The proposed algorithm is also implemented on slope stability problem to estimate the critical factor of safety. Based on the results obtained, the proposed self evolving network produced a better estimate of critical safety factor in comparison to conventional BPN network.

Keywords: Neural networks, Topology evolution, Particle swarm optimization.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1056506

References:

[1] Miller, G. F., Todd, P. M. and Hegde, S. U. (1989). "Designing neural networks using genetic algorithms." Proc. 3rd Int. Conf. Genetic Algorithms and Their Applications, J. D. Schaffer, Ed. San Mateo, CA: Morgan Kaufmann, pp. 379-384.
[2] Angeline, P. J., Saunders, G. M. and Pollack, J. B. (1994). " An evolutionary algorithm that constructs recurrent neural networks, IEEE Transactions on Neural Networks, 5, 54/65
[3] Yao, X. and Liu, Y. (1997). "A New Evolutionary System for Evolving Artificial Neural Networks." IEEE Transactions on Neural Networks, 8- 3:694-713
[4] Stanley, K. and Miikkulainen, R. (2002). "Evolving Neural Networks through Augmenting Topologies." Evolutionary Computation, 10(2): 99- 127
[5] Xian-Lun T., Yon-Guo L. and Ling Z. (2007). "A hybrid particle swarm algorithm for the structure and parameter optimization of feedforward neural networks. " LNCS 4493:213-218.
[6] Yu, J., Wang, S. and Xi, L. (2008). "Evolving artificial neural networks using an improved PSO and DPSO." Neurocomputing, 71:1054-1060
[7] Kiranyaz, S., Ince, T., Yildirim, A. and Gabbouj, M. (2009)."Evolutionary artificial neural networks by multi-dimensional particle swarm optimization." Neural Networks (2009) in press.
[8] Mat`─▒nez Garc`─▒a ,F. J. and Moreno P'erez J. A. (2008) "Jumping Frogs Optimization: a New Swarm Method for Discrete Optimization." , Technical Report DEIOC 3/2008, Department of Statistics, O.R. and Computing, University of La Laguna, Tenerife, Spain
[9] Kennedy, J. and Eberhart, R. C. (1997). "A Discrete Binary Version of the Particle Swarm Algorithm." Proceedings of IEEE Conference on Systems, Man, and Cybernetics, iscataway, New Jersey,USA. 4104- 4109.
[10] Sopena, J.M., Romero, E. and Alquezar, R. (1999); "Neural networks with periodic and monotonic activation functions: a comparative study in classification problems." Ninth International Conference on Artificial Neural Networks (ICANN ÔÇÿ99). 1:323 - 328.
[11] Wong, K.-W., Leung, C.S. and Chang, S.-J. (2002). "Use of periodic and monotonic activation functions in multilayer feedforward neural networks trained by extended Kalman filter algorithm." IEEE Proceedings. Image Signal Processing, 149 (4), 217 - 224.
[12] Jin, Y., Okabe, T. and Sendhoff, B. (2004). "Neural network regularization and ensembling using multi-objective evolutionary algorithms." Congress on Evolutionary Computation (CEC-04), IEEE
[13] Reed , R.D. and Marks, R.J. (1999). Neural Smithing. The MIT Press
[14] Narendra, K. S. and Li, S.-M.(1996). Neural networks in control systems. In P. Smolensky,M. C. Mozer, and D. E. Rumelhart, editors, Mathematical Perspectives on Neural Networks, chapter 11, pages 347- 394. Lawrence Erlbaum Associates
[15] Bishop, A .W. (1955). "The use of the slip circle in the stability analysis of slopes." Geotechnique, 5: 7-17.
[16] Janbu, N.(1973). Slope Stability Computations. Embankment Dam Engineering - Casagrande Volume, R.C. Hirschfeld and S.J. Poulos, eds., John Wiley and Sons, New York, pp 47-86.
[17] Morgensternn.,R ., and Price,V .E. (1967). "A numerical method for solving the equations of stability of general slip surfaces." Computer Journal, 9: 388-393.