Evaluation of the ANN Based Nonlinear System Models in the MSE and CRLB Senses

Authors: M.V Rajesh, Archana R, A Unnikrishnan, R Gopikakumari, Jeevamma Jacob

Abstract:

The System Identification problem looks for a suitably parameterized model, representing a given process. The parameters of the model are adjusted to optimize a performance function based on error between the given process output and identified process output. The linear system identification field is well established with many classical approaches whereas most of those methods cannot be applied for nonlinear systems. The problem becomes tougher if the system is completely unknown with only the output time series is available. It has been reported that the capability of Artificial Neural Network to approximate all linear and nonlinear input-output maps makes it predominantly suitable for the identification of nonlinear systems, where only the output time series is available. [1][2][4][5]. The work reported here is an attempt to implement few of the well known algorithms in the context of modeling of nonlinear systems, and to make a performance comparison to establish the relative merits and demerits.

Keywords: Multilayer neural networks, Radial Basis Functions, Clustering algorithm, Back Propagation training, Extended Kalmanfiltering, Mean Square Error, Nonlinear Modeling, Cramer RaoLower Bound.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1074343

References:

[1] N.K Sinha and B Kuszta, Modeling and Identification of Dynamic systems,Van Nostrand Reinhold Company, New York,1983.
[2] Simon Haykin, Neural Networks a comprehensive Foundation, Prentice Hall International Editions, 1999.
[3] A.V. Balakrishnan, Kalman Filtering Theory, Optimization Software Inc. Publications Division, Newyork,1992
[4] Fa-Long Luo and Rolf Unbehauen, Applied Neural Networks for Signal Processing, Cambridge University Press, 1997.
[5] Yaakov Bar-Shalaom and Xiao-Rong Li, Estimation and Tracking: Principles, Techniques and Software. Artech House, Boston, London, 1993.
[6] G.V. Puskorius and L.A Feldkamp, " Neuro Control of nonlinear dynamical systems with Kalman Filter trained recurrent networks" IEEE Transactions on neural networks, Vol 5, No.2, pp 279-297, 1994.
[7] K.S. Narendra and K Parthasarathy, "Identification and control of Dynamical systems using neural networks", IEEE Transactions on Neural Networks, Vol 1, No.2, pp 4-27, March 1990
[8] Shuhi Li, "Comparative Analysis of back propagation and Extended Kalman filter in Pattern and Batch forms for training Neural Networks", IEEE Transactions on Neural Network, Vol 2, No.1, pp144-149, 2001
[9] M.S. Grewal and A.P Andrews, Kalman Filtering Theory and Practice, Prentice Hall, Englewood Cliffs, 1993.
[10] Y. Linguni, H. Sakai, H. Tokumaru, "A Real Time Learning Algorithm for Multilayered Neural Network based on the Extended Kalman Filter", IEEE Transactions on Signal Processing, Vol 40, No.4, pp 959-966,1992.
[11] Joost H. de Vlieger and Robert H.J. Gmelig Meyling , "Maximum Likelihood Estimation for Long Range Target Tracking Using Passive Sonar Measurements", IEEE transactions on Signal Processing, Vol. 40, No.5, pp 1216-1225,May 1992
[12] Ivan Petrovic, Mato Baotic, Nedjeljko Peric "Model structure selection for nonlinear system identification using feed forward neural networks" ,Department of Control and Computer Engineering in Automation, Unska 3, HR - 10 000 Zagreb, Croatia.
[13] Simon Haykin, Adaptive Filter Theory, Prentice Hall International editions, 1986
[14] Anders Forsgren and Robert Kling "Implementation of Recurrent Neural Networks for Prediction and control of Nonlinear Dynamic Systems", Lulea University of Technology, Lulea, Sweden
[15] Ben James, Brian D.O, Anderson, and Robert . C. Williamson "Conditional Mean and Maximum Likelihood approaches to Multi harmonic frequency estimation.", IEEE Transactions on Signal Processing, Vol 42, No.6, pp 1366-1375, June 1994.
[16] Langford B White, "Robust Approximate likelihood ratio Tests for Nonlinear dynamic systems", IEEE Transactions on Signal Processing, Vol 43, No.8, pp 2028-2031, August 1995.