Neural Network Learning Based on Chaos

Authors: Truong Quang Dang Khoa, Masahiro Nakagawa

Abstract:

Chaos and fractals are novel fields of physics and mathematics showing up a new way of universe viewpoint and creating many ideas to solve several present problems. In this paper, a novel algorithm based on the chaotic sequence generator with the highest ability to adapt and reach the global optima is proposed. The adaptive ability of proposal algorithm is flexible in 2 steps. The first one is a breadth-first search and the second one is a depth-first search. The proposal algorithm is examined by 2 functions, the Camel function and the Schaffer function. Furthermore, the proposal algorithm is applied to optimize training Multilayer Neural Networks.

Keywords: learning and evolutionary computing, Chaos Optimization Algorithm, Artificial Neural Networks, nonlinear optimization, intelligent computational technologies.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1057557

References:

[1] Masahiro Nakagawa, Chaos and Fractals in Engineering, World Science, Singapore, 1999.
[2] Luonan Chen, K. Aihara, "Global searching ability of chaos neural networks", IEEE Trans. Circuits Syst., 46.8, 1999.
[3] Isao Tokuda, Kazuyuki Aihara, and Tomomasa Nagashima, Adaptive annealing for chaotic optimization, Phys. Rev. E, 58.4, 1998.
[4] Mohammad Saleh Tavazoei and Mohammad Haeri, "An optimization algorithm based on chaotic behavior and fractal nature", J. Comput. Appl. Math., 2006.
[5] Dixiong Yang, Gang Li and Gengdong Cheng, "On the efficiency of chaos optimization algorithms for global optimization", J. Chaos, Solitons & Fractals, 2006.
[6] Guo Zilong, Wang Sun-an and Zhuang Jian, "A novel immune evolutionary algorithm incorporating chaos optimization", Pattern Recognition Lett., 27.1, 2006,pp. 2-8.
[7] Shengsong Liu; Zhijian Hou, "Weighted gradient direction based chaos optimization algorithm for nonlinear programming problem", Proc. Intelligent Control and Automation, 3, 2002, pp. 1779 - 1783.
[8] You Yong; Sheng Wanxing; Wang Sunan; "Study of chaos genetic algorithms and its application in neural networks", Proc. Computers, Communication, Control and Power Eng.TENCON,, 1 2002. pp. 232 - 235.
[9] Rongbin Qi; Feng Qian; Shaojun Li; Zhenlei Wang; "Chaos-Genetic Algorithm for Multiobjective Optimization", Proc. Intelligent Control and Automation WCICA, 1, 2006, pp 1563 - 1566.
[10] LU Hui-juan, ZHANG Huo-ming, MA Long-hua, A new optimization algorithm based on chaos, J. Zhejiang University sci. A, 7.4, 2006, pp. 539-542.
[11] Liu Shengsong; Hou Zhijian; Wang Min, "A hybrid algorithm for optimal power flow using the chaos optimization and the linear interior point algorithm", Proc. PowerCon, 2, 2002, pp. 793 - 797
[12] Ji MJ, Tang HW, "Application of chaos in simulated annealing", J. Chaos, Solitons & Fractals, 21, 2004, pp. 933-41.
[13] Liu B, Wang L, Yin HY, Tang F, Huang DX, "Improved particle swarm optimization combined with chaos", J. Chaos, Solitons & Fractals 25, 2005, pp. 1261-71.
[14] Behera, L., Kumar, S., Patnaik, A., "On Adaptive Learning Rate That Guarantees Convergence in Feedforward Networks", IEEE Trans. Neural Networks, 17.5, 2006, pp. 1116-1125.
[15] Yu, X., Onder Efe, M., Kaynak, O., "A backpropagation learning framework for feedforward neural networks", IEEE Int. Syposium Circuits Sys. ISCAS, 3, 2001, pp.700-702.
[16] Terrence L. Fine, Feedforward neural network methodology, Springer, New York, 1999, pp. 130-131.
[17] K. Bertels1, L. Neuberg1, S. Vassiliadis and D.G. Pechanek, "On Chaos and Neural Networks: The Backpropagation Paradigm", Artificial Intelligence Rev. 15, 2001, pp.165-187.