Hopfield Network as Associative Memory with Multiple Reference Points

Authors: Domingo López-Rodríguez, Enrique Mérida-Casermeiro, Juan M. Ortiz-de-Lazcano-Lobato

Abstract:

Hopfield model of associative memory is studied in this work. In particular, two main problems that it possesses: the apparition of spurious patterns in the learning phase, implying the well-known effect of storing the opposite pattern, and the problem of its reduced capacity, meaning that it is not possible to store a great amount of patterns without increasing the error probability in the retrieving phase. In this paper, a method to avoid spurious patterns is presented and studied, and an explanation of the previously mentioned effect is given. Another technique to increase the capacity of a network is proposed here, based on the idea of using several reference points when storing patterns. It is studied in depth, and an explicit formula for the capacity of the network with this technique is provided.

Keywords: Associative memory, Hopfield network, network capacity, spurious patterns.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1332728

References:

[1] J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proc. Nat. Acad. Sci. USA, vol. 79, 1982, pp. 2554-2558.
[2] D. O. Hebb, The Organization of Behavior, New York. Wiley, 1949.
[3] R. J. McEliece, E. C. Posner, E. R. Rodemich, S. S. Venkatesh, The Capacity of the Hopfield Associative Memory, IEEE Transactions on Information Theory, vol. IT-33, no. 4, 1987, pp. 461-482.
[4] J. Hertz, A. Krogh and R. G. Palmer, Introduction to the theory of neural computation, Lecture Notes Volume I. Addison Wesley, 1991.
[5] E. Mérida-Casermeiro and J. Mu├▒oz-Pérez, MREM: An associative autonomous recurrent network, Journal of Intelligent and Fuzzy Systems 12 (3-4), pp. 163-173, 2002.