A Probabilistic View of the Spatial Pooler in Hierarchical Temporal Memory

Authors: Mackenzie Leake, Liyu Xia, Kamil Rocki, Wayne Imaino

Abstract:

In the Hierarchical Temporal Memory (HTM) paradigm the effect of overlap between inputs on the activation of columns in the spatial pooler is studied. Numerical results suggest that similar inputs are represented by similar sets of columns and dissimilar inputs are represented by dissimilar sets of columns. It is shown that the spatial pooler produces these results under certain conditions for the connectivity and proximal thresholds. Following the discussion of the initialization of parameters for the thresholds, corresponding qualitative arguments about the learning dynamics of the spatial pooler are discussed.

Keywords: Hierarchical Temporal Memory, HTM, Learning Algorithms, Machine Learning, Spatial Pooler.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1338281

References:

[1] B. Bobier and M.Wirth, “Content-based image retrieval using hierarchical temporal memory,” in Proc. 16th ACM Int. Conf. on Multimedia, 2008, pp. 925-928.
[2] P. Gabrielsson, R. Konig, and U. Johansson, “Evolving hierarchical temporal memory-based trading models,” in EvoApplications 2013-Applications of Evolutionary Computing, Vienna, April 3-5, 2013.
[3] J. Hawkins, S. Ahmad, and D. Dubinsky, “Hierarchical temporal memory including HTM cortical learning algorithms,” Numenta, Redwood City, CA, Tech. Rep. ver. 0.2.1, 2011.
[4] D.O. Hebb, ”The first stage of perception: growth of the assembly,” in The Organization of Behavior, New York, Wiley, 1949, intro. and ch. 4, pp. xi-xix, 60-78.
[5] D. Maltoni, “Pattern recognition by hierarchical temporal memory,” DEIS Univ. Bologna, Tech. Rep., pp. 1-46, Apr. 13, 2011.
[6] V. Mountcastle, “The columnar organization of the neocortex,” Brain, vol. 120(4), pp. 701-722, 1997.
[7] A.J. Perea, J.E. Merono, and M.J. Aguilera, “Application of Numenta hierarchical temporal memory for land-use classification,” S. Afr. J. Sci., vol. 105, pp. 370-375, Sept./Oct. 2009.
[8] J. Thornton and A. Srbic, “Spatial pooling for greyscale images,” Int. J. Mach. Learn. & Cyber., vol. 4, pp. 207-216, 2013.
[9] J. van Doremalen and L. Boves, “Spoken digit recognition using a hierarchical temporal memory,” Interspeech, pp. 2566-2569, Brisbane, Sept. 22-26, 2008.