Increasing The Speed of Convergence of an Artificial Neural Network based ARMA Coefficients Determination Technique

Authors: Abiodun M. Aibinu, Momoh J. E. Salami, Amir A. Shafie, Athaur Rahman Najeeb

Abstract:

In this paper, novel techniques in increasing the accuracy and speed of convergence of a Feed forward Back propagation Artificial Neural Network (FFBPNN) with polynomial activation function reported in literature is presented. These technique was subsequently used to determine the coefficients of Autoregressive Moving Average (ARMA) and Autoregressive (AR) system. The results obtained by introducing sequential and batch method of weight initialization, batch method of weight and coefficient update, adaptive momentum and learning rate technique gives more accurate result and significant reduction in convergence time when compared t the traditional method of back propagation algorithm, thereby making FFBPNN an appropriate technique for online ARMA coefficient determination.

Keywords: Adaptive Learning rate, Adaptive momentum, Autoregressive, Modeling, Neural Network.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1058333

References:

[1] Z. P. Liang, P. C. Lauterbur, "Principles of Magnetic Resonance Imaging, A signal processing perspective", IEEE Press, New York, 2000.
[2] A. M. Aibinu, M. J.E. Salami, A. A. Shafie and A. R. Najeeb, " Model Order Determination for MRI Signal", accepted for publication , International Conference on Medical system Engineering (ICMSE), 2008, Singapore, August - September 2008.
[3] D. G. Nishimura, "Principles of Magnetic Resonance Imaging", April 1996.
[4] M. R. Smith, S. T. Nichols, R. M. Henkelman and M. L. Wood, "Application of Autoregressive Moving Average Parametric Modeling in Magnetic Resonance Image Reconstruction", IEEE Transactions on Medical Imaging, Vol. M1-5:3, pp 257 - 261, 1986.
[5] M. R. Smith, S. T. Nichols, R. Constable and R. Henkelman, "A quantitative comparison of the TERA modeling and DFT magnetic resonance image reconstruction techniques", Magn. Reson. Med., Vol. 19 pp. 1-19, 1991.
[6] K. H. Chon, R. J. Cohen, "Linear and Non-Linear ARMA Model Parameter Estimation Using Artificial Neural Network", IEEE Transactions on BioMedical Engineering, Vol. 44, No 3, pp 168 - 174, 1997.
[7] K. H. Chon, D. Hoyer, A. A Armoundas, N-H Holstein-Rathlou and D. J Marsh, " Robust Nonlinear Autoregressive Moving Average Model Parameter Estimation Using Recurrent Artificial Neural Network", Annals of BioMedical Engineering, Vol. 27, pp 538-547, 1999.
[8] A. M. Aibinu, M. J. E. Salami, A. A. Shafie and A. R. Najeeb "Performance Evaluation of Autoregressive Moving Average (ARMA) coefficients determination methods", accepted for publication , International Conference on Computer System (ICCS) , 2008, Singapore, August- September 2008.
[9] M. H. Hayes, "Staitical Digital Signal processing and Modelling", John Wiley & Sons, Canada, 1996.
[10] E. C. Whitman, "The spectral analysis of discrete time series in terms of linear regressive models", Naval Ordinance Labs Rep., NOLTR-070- 109, White Oak, MD, June 23, 1974.
[11] Z. P. Liang, F. E. Boada, R. T. Constable, E. M. Haacke, P. C. Lauterbur, and M. R. Smith, "Constrained Reconstruction Methods in MR Imaging", Reviews of MRM, vol. 4, pp.67 - 185, 1992.
[12] E. Hackle and Z. Liang, "Superresolution Reconstruction Through Object Modeling and Estimation", IEEE transactions in A.S.S.P, 37: 592 - 595, 1989.
[13] R. Palaniappan, "Towards Optimal Model Oreder Selection for Autoregressive Spectral Analysis of Mental Tasks Using Genetic Algorithm", IJCSNS International Journal of Computer Science and Network Security, Vol. 6 No. 1A, January 2006.
[14] C. C. Yu, Bin-Da Liu "A Simple Procedure in Back Propagation Trainning", IEEE Trans. Autom. Control, vol. AC-19, pp. 529-535, 2001.
[15] C. C. Yu, Bin-Da Liu "A Backpropapgation Algorithm With Adaptive Learning Rate and Momentum coefficients", IEEE Trans. Autom. Control, pp. 1218-1223, 2002.
[16] S. Haykin, " Neural Networks: A comprehensive foundation, 2nd ed.", Eaglewood, Cliffs, NJ: Prentice Hall.
[17] D. Nguyen, B. Widrow, " Improving the learning speed of 2- Layer Neural Network by Choosing initial values of the adaptive weights " Proc. Int. Joint Conference on Neural Networks, Vol. 3, pp.21-26, July, 1990.
[18] J. Rissanen, "Modelling by shortest data description", Automatica, vol.14, pp.465-471, 1978.
[19] M.J. Salami, A. R. Najeeb, O. Khalifa, K. Arrifin, "MR Reconsturction with Autoregressive Moving Average", International Conference on Biotechnology Engineering, Kuala Lumpur, pp 676 - 704, May, 2007.