Hierarchical Temporal Memory (HTM)

In this 2009 ACM talk by Subutai Ahmed on HTM, he says that:

1. Mathematical formalism of HTM is done by Stanford student, Dileep George, who is also founder of Vicarious Systems.
2. HTM models or simulates the neocortex of the brain
3. Neocortex has:
   1. Many different regions performing specialised functions &
   2. Local structure is similar across regions.
4. Brain connections are roughly determined genetically but specifics are dependent on data.
5. Connections between levels or regions are bi-directionally.
6. Each region is modelling the temporally dynamics of the world.
7. Sub-cortical region Thalamus - central to attention & is kind of 'middleware for the cortex' since it connects many regions
8. Two Neocortex models - Yuval-Greenberg & Felleman

HTM Basics

1. Network of learning nodes
2. Each node: some region of cortex
3. Nodes learn common spatial patterns & common sequences( group patterns with common cause).
4. Create hierarchical, spatio-temporal model of data
5. Prob. of sequences passed up
6. Predicted spatial patterns passed down
7. Bayesian method resolve ambiguity
8. HTM node: memorizes static patterns or coincidences, stable sparse vectors
9. Model freq of transition between patterns: temporal, variable order markov chains
10. Hierarchy allows contextual prediction

Application: Web Analytics

1. Forbes.com web analytics: predict topic & pages an individual user might be interested in.
2. 177 total topics
3. Random prediction: 0.56%
4. HTM trained on 100K user sequences - each containing 6-7 clicks
5. Possible approaches:
   1. Pageview stats: 0-th order prediction & no temporal aspect in it.
   2. 1st-order prediction: model transition prob from one page to another
   3. Variable order prediction: jumps to 45%
6. Batch training was done, incremental training is tricky -
   1. top-level does get confused by changing sequence of sequences from lower layers

Application: Others

1. Person recognition from Video camera surveillance images
2. Bio-medical Imaging: specific to prostate cancer screening: glands & non-glands
   1. Accuracy was 95% (from 80%) from existing training set
3. Credit-card fraud
4. Drug discovery
5. Gaming
6. Business modelling: predicting inventory, uptake dep. on marketing campaign
7. Healthcare
8. Automotive: pedestrian detection
9. Vision toolkit beta: free for research - NOT SURE if still available

Competing Companies

1. Columbia Univ. Bionet Group, Olfactory modeling on GPU, in Menlopark
2. Blugene project of IBM

Numenta Videos on HTM & CLA

1. CLA Deep Dive (2013 Fall NuPIC Hackathon), Subutai refers to Bloom Filter but in the CLA Quiz video, he says that the Bloom filter has random outputs which is not what we want.
2. CLA basics - Part 1

Other Software

1. The S-Space Package: A scalable software library for semantic spaces
2. Topic modeling for humans, Python library
3. Parallelizing word2vec in Python, Oct 2013
4. A C++ Library for Discrete Graphical Models
5. [[|]]

HTM & CLA for Natural Language Processing (NLP)

In this talk by Franscisco Webber on their CELP API implementation, he refers to Word-space model, semantic spaces, random indexing & Google's wordvec software.

Measuring the dimensionality of general metric spaces

Google search for [time-series data analysis topology] has 330 unique hits only.