Investigation of Improved Chaotic Signal Tracking by Echo State Neural Networks and Multilayer Perceptron via Training of Extended Kalman Filter Approach

Authors: Farhad Asadi, S. Hossein Sadati

Abstract:

This paper presents a prediction performance of feedforward Multilayer Perceptron (MLP) and Echo State Networks (ESN) trained with extended Kalman filter. Feedforward neural networks and ESN are powerful neural networks which can track and predict nonlinear signals. However, their tracking performance depends on the specific signals or data sets, having the risk of instability accompanied by large error. In this study we explore this process by applying different network size and leaking rate for prediction of nonlinear or chaotic signals in MLP neural networks. Major problems of ESN training such as the problem of initialization of the network and improvement in the prediction performance are tackled. The influence of coefficient of activation function in the hidden layer and other key parameters are investigated by simulation results. Extended Kalman filter is employed in order to improve the sequential and regulation learning rate of the feedforward neural networks. This training approach has vital features in the training of the network when signals have chaotic or non-stationary sequential pattern. Minimization of the variance in each step of the computation and hence smoothing of tracking were obtained by examining the results, indicating satisfactory tracking characteristics for certain conditions. In addition, simulation results confirmed satisfactory performance of both of the two neural networks with modified parameterization in tracking of the nonlinear signals.

Keywords: Feedforward neural networks, nonlinear signal prediction, echo state neural networks approach, leaking rates, capacity of neural networks.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.2022091

References:

[1] R. Laje, D. V. Buonomano, "Robust timing and motor patterns by taming chaos in recurrent neural networks," Nature neuroscience, 2013, 16(7), pp. 925.
[2] M. Lukoševičius, "A practical guide to applying echo state networks," Neural networks: Tricks of the trade: Springer, 2012. pp. 659-86.
[3] F. Asadi, M. J. Mollakazemi, S. A. Atyabi, I. Uzelac, A. Ghaffari, "Cardiac arrhythmia recognition with robust discrete wavelet-based and geometrical feature extraction via classifiers of SVM and MLP-BP and PNN neural networks," in Computing in Cardiology Conference (CinC), 2015 2015, IEEE.
[4] F. Asadi, M. J. Mollakazemi, S. Ghiasi, S. H. Sadati, "Enhancement of life-threatening arrhythmia discrimination in the intensive care unit with morphological features and interval feature extraction via random forest classifier," in Computing in Cardiology Conference (CinC), 2016 2016, IEEE.
[5] S. A. S. Mousavi, X. Zhang, T. Seigler, J. B. Hoagg, "Characteristics that make dynamic systems difficult for a human to control," in American Control Conference (ACC), 2016 2016, IEEE.
[6] F. Matveeva, S. A. S. Mousavi, X. Zhang, T. Seigler, J. B. Hoagg, "On the effects of changing reference command as humans learn to control dynamic systems," in Decision and Control (CDC), 2016 IEEE 55th Conference on 2016, IEEE.
[7] A. Hamidisepehr, M. P. Sama, "A low-cost method for collecting hyperspectral measurements from a small unmanned aircraft system," in Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping III 2018, International Society for Optics and Photonics.
[8] A. Hamidisepehr, M. P. Sama, A. P. Turner, O. O. Wendroth, "A Method for Reflectance Index Wavelength Selection from Moisture- Controlled Soil and Crop Residue Samples," Transactions of the ASABE, 2017, 60(5), pp. 1479-87.
[9] A. Ghaffari, M. J. Mollakazemi, S. A. Atyabi, M. Niknazar, "Robust fetal QRS detection from noninvasive abdominal electrocardiogram based on channel selection and simultaneous multichannel processing," Australasian physical & engineering sciences in medicine, 2015, 38(4), pp. 581-92.
[10] M. Mollakazemi, F. Asadi, M. Tajnesaei, A. Ghaffari, "Fetal QRS Detection in Noninvasive Abdominal Electrocardiograms Using Principal Component Analysis and Discrete Wavelet Transforms with Signal Quality Estimation," Journal of Biomedical Physics and Engineering, 2016, pp.
[11] M. J. Mollakazemi, S. A. Atyabi, A. Ghaffari, "Heart beat detection using a multimodal data coupling method," Physiological measurement, 2015, 36(8), pp. 1729.
[12] D. Brezak, T. Bacek, D. Majetic, J. Kasac, B. Novakovic, "A comparison of feed-forward and recurrent neural networks in time series forecasting," in Computational Intelligence for Financial Engineering & Economics (CIFEr), 2012 IEEE Conference on 2012, IEEE.
[13] M. Khashei, M. Bijari, "A novel hybridization of artificial neural networks and ARIMA models for time series forecasting," Applied Soft Computing, 2011, 11(2), pp. 2664-75.
[14] A. Petrenas, V. Marozas, L. Sörnmo, A. Lukosevicius, "An echo state neural network for QRST cancellation during atrial fibrillation," IEEE Transactions on Biomedical Engineering, 2012, 59(10), pp. 2950.
[15] F. Asadi, M. Khorram, S. A. A. Moosavian, "CPG-based gait planning of a quadruped robot for crossing obstacles," in Robotics and Mechatronics (ICROM), 2015 3rd RSI International Conference on 2015, IEEE.
[16] F. Asadi, M. Khorram, S. A. A. Moosavian, "CPG-based gait transition of a quadruped robot," in Robotics and Mechatronics (ICROM), 2015 3rd RSI International Conference on 2015, IEEE.
[17] B. Choubin, S. Khalighi-Sigaroodi, A. Malekian, Ö. Kişi, "Multiple linear regression, multi-layer perceptron network and adaptive neurofuzzy inference system for forecasting precipitation based on large-scale climate signals," Hydrological Sciences Journal, 2016, 61(6), pp. 1001- 9.
[18] D. Sussillo, "Neural circuits as computational dynamical systems," Current opinion in neurobiology, 2014, 25pp. 156-63.
[19] D. Sussillo, M. M. Churchland, M. T. Kaufman, K. V. Shenoy, "A neural network that finds a naturalistic solution for the production of muscle activity," Nature neuroscience, 2015, 18(7), pp. 1025.
[20] M. Lukoševičius, H. Jaeger, "Reservoir computing approaches to recurrent neural network training," Computer Science Review, 2009, 3(3), pp. 127-49.
[21] Z. Shi, M. Han, "Support vector echo-state machine for chaotic timeseries prediction," IEEE Trans Neural Networks, 2007, 18(2), pp. 359- 72.
[22] M. Čerňanský, P. Tiňo, "Predictive modeling with echo state networks," in International Conference on Artificial Neural Networks 2008, Springer.
[23] Y. Xia, B. Jelfs, M. M. Van Hulle, J. C. Príncipe, D. P. Mandic, "An augmented echo state network for nonlinear adaptive filtering of complex noncircular signals," IEEE Transactions on Neural Networks, 2011, 22(1), pp. 74-83.
[24] M. J. Mollakazemi, F. Asadi, A. Ghafouri, "The evaluation of the performance of different filtering approaches in tracking problem and the effect of noise variance," simulation, 2015, 10pp. 12.
[25] F. Asadi, M. J. Mollakazemi, "Investigation on Performance of Change Point Algorithm in Time Series Dynamical Regimes and Effect of Data Characteristics," World Academy of Science, Engineering and Technology, International Journal of Mechanical, Aerospace, Industrial, Mechatronic and Manufacturing Engineering, 2015, 8(10), pp. 1787-93.
[26] F. Asadi, M. Mollakazemi, A. Ghafouri, "The influence of parameters of modeling and data distribution for optimal condition on locally weighted projection regression method," Accepted and oral presentation in ICMSE, 2014, pp. 27-8.
[27] M. J. Mollakazemi, F. Asadi, "Real Time Adaptive Obstacle Avoidance in Dynamic Environments with Different DS," Accepted and oral presentation in ICARM, 2014, pp. 27-8.