Towards Growing Self-Organizing Neural Networks with Fixed Dimensionality

Authors: Guojian Cheng, Tianshi Liu, Jiaxin Han, Zheng Wang

Abstract:

The competitive learning is an adaptive process in which the neurons in a neural network gradually become sensitive to different input pattern clusters. The basic idea behind the Kohonen-s Self-Organizing Feature Maps (SOFM) is competitive learning. SOFM can generate mappings from high-dimensional signal spaces to lower dimensional topological structures. The main features of this kind of mappings are topology preserving, feature mappings and probability distribution approximation of input patterns. To overcome some limitations of SOFM, e.g., a fixed number of neural units and a topology of fixed dimensionality, Growing Self-Organizing Neural Network (GSONN) can be used. GSONN can change its topological structure during learning. It grows by learning and shrinks by forgetting. To speed up the training and convergence, a new variant of GSONN, twin growing cell structures (TGCS) is presented here. This paper first gives an introduction to competitive learning, SOFM and its variants. Then, we discuss some GSONN with fixed dimensionality, which include growing cell structures, its variants and the author-s model: TGCS. It is ended with some testing results comparison and conclusions.

Keywords: Artificial neural networks, Competitive learning, Growing cell structures, Self-organizing feature maps.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1334197

References:

[1] B. Fritzke, Some competitive learning methods. http://www.neuroinformatik..ruhr-uni-bochum.de/ini/VDM/research/gsn/ JavaPaper/
[2] T. Kohonen, Self-Organizing Maps. Springer, Berlin, Heidelberg, 2001. (Third Extended Edition).
[3] Fritzke, Growing cell structures - a self-organizing network for unsupervised and supervised learning. Neural Networks, 7(9):1441-1460, 1994.
[4] A. Zell, M. Schmalzl, Dynamic LVQÔÇöA fast neural net learning algorithm. In Proc. ICANN-94, International Conference on Artificial Neural Networks, volume II, pages 1095-1098, London, UK, 1994.
[5] J. Goppert, W. Rosenstiel, Interpolation in SOM: Improved generalization by iterative methods. In Proc. ICANN-95, International Conference on Artificial Neural Networks, pages 69-74, France, 1995.
[6] A. Zell, H. Bayer, H., Bauknecht, Similarity analysis of molecules with self-organizing surfacesÔÇöan extension of the self-organizing map. In Proc. ICNN-94, International Conference on Neural Networks, pages 719-724, Piscataway, NJ, 1994. IEEE Service Center.
[7] D. Deng, N. Kasabov, On-line pattern analysis by evolving self-organizing maps Neurocomputing 51: 87-103 (2003)
[8] J. Blackmore, Visualizing high-dimensional structure with the incremental grid growing neural network. Technical Report AI95-238, University of Texas, Austin, August 1, 1995.
[9] M. Dittenbach, F. Merkl, A. Rauber, The growing hierarchical self-organizing map. In S. Amari, C. L. Giles, M. Gori, and V. Puri, editors, Proc of the International Joint Conference on Neural Networks (IJCNN 2000), volume VI, pages 15 - 19, Como, Italy, July 24. - 27. 2000. IEEE Computer Society.
[10] J. Bruske, G. Sommer, Dynamic cell structure learns perfectly topology preserving map. Neural Computation, 7(4):845-865, 1995.
[11] T. Martinetz, Competitive Hebbian learning rule forms perfectly topology preserving maps. In Stan Gielen and Bert Kappen, editors, Proc. ICANN-93, Int. Conf. on Artificial Neural Networks, pages 427-434, London, UK, 1993. Springer.
[12] V. Hodge, J. Austin, Hierarchical growing cell structures: TreeGCS. In IEEE TKDE Special Issue on Connectionist Models for Learning in Structured Domains.
[13] N. Vlassis, A. Dimopoulos, G. Papakonstantinou, The probabilistic growing cell structures algorithm. Lecture Notes in Computer Science, 1327:P649, 1997.
[14] J.Pakkanen, J. Iivarinen, E. Oja, The evolving tree -- a novel self-organizing network for data analysis. Neural Processing Letters, 20(3):199{211, 2004.
[15] Y. Wang, C. Yang, K. Mathee, G. Narasimhan, Clustering using Adaptive Self-Organizing Maps (ASOM) and Applications. Proceedings of International Workshop on Bioinformatics Research and Applications, p944-951 Atlanta, Georgia, May 2005.
[16] R. Freeman and H. Yin, "Adaptive Topological Tree Structure (ATTS) for document organization and visualization," Neural Networks, Vol. 17, pp. 1255-1271, 2004.
[17] M. White, S. Fahlman, CMU repository of neural network benchmarks. Technical report, The Carnegie Mellon University, 1997.