Towards a Mathematical Theory of Cortical Micro-circuits

By Dileep George, Jeff Hawkins. 2009

446 citations recorded by the source corpus.

The theoretical setting of hierarchical Bayesian inference is gaining acceptance as a framework for understanding cortical computation. In this paper, we describe how Bayesian belief propagation in a spatio-temporal hierarchical model, called Hierarchical Temporal Memory (HTM), can lead to a mathematical model for cortical circuits. An HTM node is abstracted using a coincidence detector and a mixture of Markov chains. Bayesian belief propagation equations for such an HTM node define a set of functional constraints for a neuronal implementation. Anatomical data provide a contrasting set of organizational constraints. The combination of these two constraints suggests a theoretically derived interpretation for many anatomical and physiological features and predicts several others. We describe the pattern recognition capabilities of HTM networks and demonstrate the application of the derived circuits for modeling the subjective contour effect. We also discuss how the theory and the circuit can be extended to explain cortical features that are not explained by the current model and describe testable predictions that can be derived from the model.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Neuroscience

Citations in the corpus

What can the brain teach us about building artificial intelligence? (2016, background): “There is growing evidence for the use of MP-like inference in cortical areas [7, 8] and they could offer a happy medium where inference is fast as in neural networks, while retaining MCMC’s capability for answering arbitrary queries on the model.”
Unsupervised Continual Learning and Self-Taught Associative Memory Hierarchies (2019, background): “Even though our high-level position is the same (namely, that the basic building block of artiﬁcial neural networks should be a functional model of the cortical column, rather than individual neurons), the Numenta architecture (“Hierarchical Temporal Memory”) is fundamentally different than the STAM”
Canonical microcircuits for predictive coding. (2012, unknown): “More Neuron 76, November 21, 2012 ª2012 Elsevier Inc. 701 recently, George and Hawkins have suggested that the canonical microcircuit implements a form of Bayesian processing (George and Hawkins, 2009).”
New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative (2010, background): “This brings us close to a number of cognition-inspired concept detection mechanisms, such as [27] and [9].”
Active Inference: A Process Theory (2017, background): “Key among these are hierarchical models with deep temporal structure (George & Hawkins, 2009; Specht, 2014), and models in which prior preferences are absorbed into beliefs about state transitions or contingencies.”
The atoms of neural computation (2014, background): “While we will not attempt a comprehensive survey here, several theories (Anselmi et al., 2013; Bhand, Mudur, Suresh, Saxe, & Ng, 2011; George & Hawkins, 2009; Mallat, 2013; Poggio, Mutch, Leibo, Rosasco, & Tacchetti, 2012) aim to give a general account of a broad range of hierarchical sensory…”
A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs (2017, background): “We hope that this work inspires improved models of cortical circuits (61, 62) and investigations that combine the power of neural networks and structured probabilistic models toward general artificial intelligence systems.”
Semi-supervised tuning from temporal coherence (2015, background): “Hierarchical temporal memory (HTM) (George & Hawkins (2009)) is a biologically inspired framework that can be framed into multistage Hubel-Wiesel architectures (Ranzato et al. (2007)), a spe-ciﬁc family of deep architectures.”
Selectionist and Evolutionary Approaches to Brain Function: A Critical Appraisal (2012, background): “George and Hawkins (2009) present a fairly detailed, but tentative account in terms of cortical microcircuits.”