Application of Extreme Learning Machine Method for Time Series Analysis

Authors: Rampal Singh, S. Balasundaram

Abstract:

In this paper, we study the application of Extreme Learning Machine (ELM) algorithm for single layered feedforward neural networks to non-linear chaotic time series problems. In this algorithm the input weights and the hidden layer bias are randomly chosen. The ELM formulation leads to solving a system of linear equations in terms of the unknown weights connecting the hidden layer to the output layer. The solution of this general system of linear equations will be obtained using Moore-Penrose generalized pseudo inverse. For the study of the application of the method we consider the time series generated by the Mackey Glass delay differential equation with different time delays, Santa Fe A and UCR heart beat rate ECG time series. For the choice of sigmoid, sin and hardlim activation functions the optimal values for the memory order and the number of hidden neurons which give the best prediction performance in terms of root mean square error are determined. It is observed that the results obtained are in close agreement with the exact solution of the problems considered which clearly shows that ELM is a very promising alternative method for time series prediction.

Keywords: Chaotic time series, Extreme learning machine, Generalization performance.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1078657

References:

[1] P. J. Brockwell and R. A. Davis, "Introduction to Time Series Forecasting", 2nd ed., Springer, Berlin, 2002.
[2] M.Casdagli, "Nonlinear Prediction of Chaotic Time Series", Physica D, 35, (1989), pp. 335-356.
[3] K. Y. Chen and C. H. Wang, "A Hybrid SARIMA and Support Vector Machines for Forecasting the Production Values of the Machinery Industry in Taiwan", Expert Systems with Applications, (2006).
[4] Y. Chen, B.Yang and J.Dong, "Time Series Prediction using a Local Linear Wavelet Neural Network", Neurocomputing, 69 (2006), pp.449- 465.
[5] G. B. Huang, Q. Y. Zhu and C. K. Siew, "Extreme Learning Machine: Theory and Applications", Neurocomputing, 70, (2006), pp.489-501.
[6] R.Malhotra and D.K.Malhotra, "Evaluating Consumer Loans Using Neural Networks", Omega, 31, (2003), pp.83-96.
[7] N.Mani and P.Voumard, "An Optical Character Recognition Using Artificial Neural Network", IEEE Int. Conf. on Systems, Man, and Cybernetics, Vol. 3, (1996), pp.2244-2247.
[8] S.Mukherjee, E.Osuna and F.Girosi, "Nonlinear Prediction of Chaotic Time Series Using Support Vector Machines", in Neural Networks for Signal Processing VII, Proceed. of the IEEE Signal Processing Society Workshop, FL, (1997), pp.511-520.
[9] K.R.Muller, A.J.Smola, G.Ratsch, B.Schlkopf and J.Kohlmorgen, "Using Support Vector Machines for Time Series Prediction", in B.Schlkopf, C.J.C. Burges and A.J.Smola (Eds), Advances in Kernel Methods- Support Vector Learning, MIT Press, Cambridge, MA, (1999), pp.243-254.
[10] C.R.Rao and S.K.Mitra, Generalized Inverse of Matrices and its Applications, Wiley, New York, (1971).
[11] H.A.Rowley, S.Baluja and T.Kanade, "Neural Network based Face Detection", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.20, No. 1, (1998), pp.23-38.
[12] Z.Tang, P.A.Fishwick, "Feedforward Neural Nets as Models for Time Series Forecasting", ORSA J. Comput. 5(1993), pp.374-385.
[13] F.E.H.Tay and L.Cao, "Application of Support Vector Machines in Financial Time Series Forecasting", Omega 29 (2001), pp.309-317.
[14] Q.Tong, H.Zheng and X.Wang, "Gene Prediction Algorithm Based on the Statistical Combination and the Classification in terms of Gene Characteristics", Int. Conf. on Neural Networks and Brain, Vol.2, (2005), pp.673 - 677.
[15] T.B.Trafalis, H.Ince, "Support Vector Machine for Regression and Applications to Financial Forecasting", Proceedings of the IEEE INNSENNS Int. Joint Conf., Vol.16, IEEE (2000), pp. 348-353.
[16] F.M.Tseng, H.C.Yu and G.H.Tzeng, "Combining Neural Network Model with Seasonal Time Series ARIMA Model", Technological Forecasting and Social Change, 69 (2002), pp.71-87.
[17] G.P.Zhang, E.B.Patuwo and M.Y.Hu, "A Simulation Study of Artificial Neural Networks for Nonlinear Time Series Forecasting", Comput.Oper.Res. 28,(2001), pp.381-396.
[18] G.P.Zhang, "Time Series Forecasting using a Hybrid ARIMA and Neural Network Model", Neurocomputing, 50, (2003),pp. 159-175.