Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons

By Dejan Pecevski, Lars Buesing, Wolfgang Maass. 2011

133 citations recorded by the source corpus.

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows (“explaining away”) and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons.

Subfields: Artificial Intelligence, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

Towards Biologically Plausible Deep Learning (2015, background): “, via a Markov chain which may correspond to probabilistic inference with spiking neurons (Pecevski et al., 2011).”
Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity (2013, background): “In particular, [24,25] have recently shown how recurrently connected neurons can be designed to perform neural sampling, an approach in which time-independent probability distributions can be represented through spiking activity in recurrent neural networks.”
Noise as a Resource for Computation and Learning in Networks of Spiking Neurons (2014, methodology): “This situation is often described in terms of the ‘‘explaining away effect,’’ which occurs, for example, in context dependence perception tasks (see [13]), and can, therefore, not be ignored for real-world applications.”
Stochastic variational learning in recurrent spiking networks (2014, background): “Moreover, it has been suggested recently (Pecevski et al., 2011; Shao, 2012) that spiking networks with biologically plausible dynamics can produce stationary samples from Deep Boltzmann Machines (Salakhutdinov and Hinton, 2009) and more general Bayesian networks.”
Opportunities for neuromorphic computing algorithms and applications (2022, methodology): “Pecevski et al. 101 used neuromorphic hardware to perform inference and sampling on general graphical structures, such as Bayes’s nets, which is NP complete for random variables with probabilities not bounded away from zero 102 .”