Heuristic Continuous-time Associative Memories

Authors: Truong Quang Dang Khoa, Masahiro Nakagawa

Abstract:

In this paper, a novel associative memory model will be proposed and applied to memory retrievals based on the conventional continuous time model. The conventional model presents memory capacity is very low and retrieval process easily converges to an equilibrium state which is very different from the stored patterns. Genetic Algorithms is well-known with the capability of global optimal search escaping local optimum on progress to reach a global optimum. Based on the well-known idea of Genetic Algorithms, this work proposes a heuristic rule to make a mutation when the state of the network is trapped in a spurious memory. The proposal heuristic associative memory show the stored capacity does not depend on the number of stored patterns and the retrieval ability is up to ~ 1.

Keywords: Artificial Intelligent, Soft Computing, NeuralNetworks, Genetic Algorithms, Hopfield Neural Networks, andAssociative Memories.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1332214

References:

[1] Masahiro Nakagawa, Chaos and Fractals in Engineering, World Scientific, Singapore, 1999.
[2] Mohamad H. Hassoun, Associative Neural Memories: Theory and Implementation, Oxford University Press, USA 1993.
[3] S. Haykin, Neural Networks: A Comprehensive Foundation, Second Edition, Macmillan, New York, 1999.
[4] C. M. Marcus and R. M. Westervelt, "Stability and convergence of analog neural networks with multiple-time-step parallel dynamics", Phys. Rev. A 42, 1990, pp.2410
[5] F. R. Waugh, and R. M. Westervelt, "Associative memory in an analog iterated-map neural network C. M. Marcus", Phys. Rev. A 41, 1990, pp.3355
[6] Lee, R.S.T., "A transient-chaotic autoassociative network (TCAN) based on Lee oscillators", IEEE Trans. Neural Networks, Vol. 15, No. 5, Sep. 2004, pp.1228 - 1243
[7] Yanai, H.-F., and Amari, S.-I., "Auto-associative memory with two-stage dynamics of nonmonotonic neurons", IEEE Trans. Neural Networks, Vol. 7, No. 4, July 1996, pp.803 - 815
[8] Seiji Miyoshi, and Masato Okada, "Storage Capacity Diverges With Synaptic Efficiency in an Associative Memory Model With Synaptic Delay and Pruning", IEEE Trans. Neural Networks, Vol. 15, No. 5, Sep. 2004, pp.1215
[9] Masaharu Adachi and Kazuyuki Aihara, "Associative Dynamics in a Chaotic Neural Network", J. Neural Networks, Vol. 10, No. 1, Jan. 1997, pp. 83-98
[10] Masahiko Morita, "Associative memory with nonmonotone dynamics", J. Neural Networks, Vol. 6, No. 1, 1993, pp. 115-126
[11] Masahiro Nakagawa, "Statistical Properties of Chaos Associative Memory", J. of the Physical Society of Japan, Vol. 71, No. 9, Sep. 2002, pp. 2316-2325
[12] Masahiro Nakagawa, "A Chaos Associative Memory with a Skew-Tent Activation Function", J. of the Physical Society of Japan, Vol.69, No.3, Mar. 2000, pp. 972-980
[13] Masahiro Nakagawa, "Chaos Associative Memory with a Periodic Activation Function", J. of the Physical Society of Japan, Vol. 67, No. 7, July 1998, pp. 2281-2293
[14] Masahiro Nakagawa, "Entropy Based Associative Memory", IEICE Trans. Fundamentals of Electronics, Communications and Computer, Vol.E89-A, No.4, 2006, pp.895-901
[15] Truong Quang Dang Khoa, Masahiro Nakagawa, Neural Network Learning based on Chaos, WASET Inter. J. Computer, Information, and Systems Science, and Engineering, Vol. 1, No. 2, 2007, pp.97-102