Analysis of Multilayer Neural Network Modeling and Long Short-Term Memory

Authors: Danilo López, Nelson Vera, Luis Pedraza

Abstract:

This paper analyzes fundamental ideas and concepts related to neural networks, which provide the reader a theoretical explanation of Long Short-Term Memory (LSTM) networks operation classified as Deep Learning Systems, and to explicitly present the mathematical development of Backward Pass equations of the LSTM network model. This mathematical modeling associated with software development will provide the necessary tools to develop an intelligent system capable of predicting the behavior of licensed users in wireless cognitive radio networks.

Keywords: Neural networks, multilayer perceptron, long short-term memory, recurrent neuronal network, mathematical analysis.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1339748

References:

[1] Zachary, L., J. Berkowitz., C. Elkan., 2015. A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019.
[2] Pérez, J., 2002. Predictive models based on discrete time recurrent neural networks. Department of Language and Computer Systems. PhD Thesis, University of Alicante.
[3] Alex, Graves., 2012. Supervised Sequence Labelling with Recurrent Neural Networks. Poland: Springer, ISBN: 978-3642247965.
[4] Neural networks basics. (Online). Accessed on February 7 2015, retrieved from http://grupo.us.es/gtocoma/pid/pid10/RedesNeuronales.htm.
[5] Artificial neuronal networks in intensive medicine. An example of application with MPM II variables. (Online). Accessed May 17 2015, retrieved from http://www.medintensiva.org/es/redes-neuronales-artificiales-medicina-intensiva-/articulo/13071859/.
[6] Yan, S. Understanding LSTM networks. (Online). Accessed on August 11 2015 retrieved from http://colah.github.io/posts/2015-08-Understanding-LSTMs/.
[7] Schmidhuber, J., S, Hochrei., 1997. Long short-term memory. Journal Neural computation, 9: 1735-1780.
[8] Ke-Lin, D., M. Swamy., 2013. Neural networks and statistical learning. New York: Springer & Business Media, ISBN: 978-1447170471.