Independent Sites & Research Papers
Links to sites and research papers from independent scientists and engineers about Hierarchical Temporal Memory and about Numenta technology. Descriptions of the papers were written by Numenta employees.
Sites
Numenta Watch
Numenta Watch is a site in Japanese that is tracking information and events related to Numenta, HTM, and NuPIC.
ヌメンタウォッチはNumenta社やHTM関連技術、NuPICフレームワーク等の動向を日本語で提供しています。
Research Papers
Support for the Use of Hierarchical Temporal Memory Systems in Automated Design Evaluation: A First Experiment »
Josh Hartung, Jay McCormack & Frank Jacobus
In this paper, a hierarchical temporal memory system (HTM) is proposed as a viable approach for capturing design quality from exemplars and subsequently recognizing the presence of that quality in other designs. It includes a first experiment in using HTMs for learning and recognizing quality in the form of the visual style characteristics of Hepplewhite, Stickley, and Greene & Greene chair backs. Results show that HTMs develop a similar storage of quality to humans and are therefore a promising option for capturing and recognizing multi-modal quality information in future design automation projects.
Despite many years of research, computational synthesis tools that automatically generate solutions to design problems are not widely used in industry. This deficiency can be attributed to the lack of value that these tools provide for the user in terms of time saved in design or quality improvements in the design. In order to provide sufficient quality of solution, it is proposed that more human-like evaluation of solution quality is needed including qualitative concepts, the ability to allow for anecdotal input, and general inclusion of ambiguous information.
Josh Hartung, Jay McCormack & Frank Jacobus (2009). Support for the Use of Hierarchical Temporal Memory Systems in Automated Design Evaluation: A First Experiment. Proceedings of the ASME 2009 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference IDETC/CIE 2009 [copyright © ASME 2009].
Robust Character Recognition using a Hierarchical Bayesian Network »
Thornton, J. R., Gustafsson, T., Blumenstein, M. & Hine, T.
This paper gives an example of trying a part of the HTM algorithm on a different data set. The MNIST digit data set is a standard benchmark in the machine learning community to verify the performance of recognition algorithms. The authors used this data set for their experiments. The algorithm that they used did not involve the temporal component - it is a hierarchical memory system without the temporal part. Therefore, the generalization properties that we expect from temporal pooling cannot be expected from this implementation. An examination of the data set also shows that the extent of translation and scaling in this data set is very less compared to the pictures data set. Therefore, we would expect a reasonable performance without including the temporal part.
The authors tried this algorithm against other popular algorithms and obtained comparable results. Although the HTM variant does not beat the best algorithm for this purpose, that was not the objective of this paper. This paper would be interesting to anybody who is interested in knowing how to apply HTMs to the MNIST dataset. However, the reader should keep in mind that the temporal part of HTMs was not implemented in this paper.
Thornton, J. R., Gustafsson, T., Blumenstein, M. & Hine, T. (2006). Robust Character Recognition using a Hierarchical Bayesian Network. Proceedings of the 19th Australian Joint Conference on Artificial Intelligence, AI-2006, Hobart. 1259-1264. [copyright © Springer-Verlag].
Memory-Prediction Framework for Pattern Recognition: Performance and Suitability of the Bayesian Model of Visual Cortex »
Saulius J. Garalevicius
In this paper, Saulius extends the original Pictures network by training them on more categories and reports interesting experimental results on the performance of the network under various conditions. He plots the performance of the initial model as a function of number of eye movements and uses that as a benchmark while exploring some modifications to the network and the training strategy. The initial network did not have nodes with overlapping receptive fields. The authors experiment to find out the effect of overlapping on the performance was inconclusive. Although the performance was better for between 1-3 eye movements, the performance for larger number of eye movements fluctuated above and below the original model. The author also does an interesting experiment on memory usage vs recognition accuracy.
The author raises many valid concerns about the original model and many of these have been addressed in the new Pictures example. This paper should be an interesting read for anyone thinking of doing additional experiments based on the Pictures example.
Garalevicius, Saulius. (2006). Memory-Prediction Framework for Pattern Recognition . Department of Computer and Information Sciences, Temple University.
Biomimetic sensory abstraction using hierarchical quilted self-organizing maps »
J. W. Miller and P. H. Lommel
HTMs use temporal and spatial pooling at all levels of the hierarchy to obtain invariant representations. Different algorithms can be used to achieve this temporal and spatial pooling. In our current HTM implementation, we learn a temporal transition matrix and then partition the graph represented by that matrix to obtain temporal pools.
In contrast, this paper uses a Self Organizing Map to achieve temporal pooling (temporal clustering). The SOMs seem to have a closer relation to biology although they could turn out to be theoretically equivalent to temporal transition learning. Moreover, SOMs can also be used to cluster together 'similar' units in space, thus reducing the effective search space for a higher level node. In the current implementation of HTMs, the boundaries of the nodes abruptly stop the information flow. SOMs also provide a convenient way to implement overlapping nodes.
This paper also uses online learning. Our current set of algorithms do not do on-line learning. This is something we would rectify in our next version of algorithms. Disadvantages of using SOMs could include increased processing requirements and poor understanding of the convergence and scaling properties. Incorporating feedback and predicting forward in time could also prove to be tricky.
Understanding and creating efficient algorithms for temporal and spatial pooling and integrating them in a hierarchy is the fundamental process behind creating the HTM technology. This paper demonstrates the use of an alternative set of algorithms that could be used for this purpose.
Disadvantages of using SOMs could include increased processing requirements and poor understanding of the convergence and scaling properties. Incorporating feedback and predicting forward in time could also prove to be tricky.
Miller, J.W., Lommel, P.H. Biomimetic sensory abstraction using hierarchical quilted self-organizing maps. The Charles Stark Draper Laboratory, Inc.
