On the Computational Power of Winner-Take-All

By Wolfgang Maass. 2000

414 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “Instead of using alternative local subsampling or WTA methods (e.g., Fukushima, 1980, 2013a; Maass, 2000; Schmidhuber, 1989b), the Cresceptron uses Max-Pooling (MP) layers.”
Selfless Sequential Learning (2018, background): “While the idea of lateral inhibition depicted in Maxout [14] and LWTA [39] is interesting and powerful [28], it remains restricted to a fixed window size.”
Opportunities for neuromorphic computing algorithms and applications (2022, background): “Such recurrent SNNs have shown greater classification ability with winner-take-all models 76–78 .”
Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems (2014, background): “The computational abilities of these types of networks are of great importance in tasks involving feature extraction, signal restoration, and pattern classification problems [113].”
Competitive STDP-Based Spike Pattern Learning (2009, background): “It is known that in feedforward networks, no computational power is lost if, like here, inhibition is used exclusively for unspecific lateral inhibition, and no flexibility is lost if, like here, plasticity is restricted to excitatory synapses (Maass, 2000).”
Neuronal circuits of the neocortex. (2004, background): “The selection mechanism is a soft winner-take-all or soft MAX mechanism, which are important elements of many neuronal network models (Maass 2000, Riesenhuber & Poggio 1999, Yuille & Geiger 2003).”
Memory and Information Processing in Neuromorphic Systems (2015, background): “6), and have been shown to posses powerful computational properties for tasks involving feature extraction, signal restoration, and pattern classification [105].”
Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations (2002, background): “Theoretically, the class of readout maps that can be implemented in this fashion satises the approximation property AP (Maass, 2000; Auer, Burgsteiner, & Maass, 2001), and is according to theorem 1 in principle sufcient for approximating arbitrary given fading memory lters F.”
Compete to Compute (2013, background): “For example, Maas [17, 18] showed that feedforward neural networks with WTA dynamics as the only non-linearity are as computationally powerful as networks with threshold or sigmoidal gates; and, networks employing only soft WTA competition are universal function approximators.”
Nonparametric Bayesian Deep Networks with Local Competition (2018, background): “…local winner-takes-all (LWTA) networks can discover effective sparsely distributed representations of their input stimuli (Lee & Seung, 1999; Olshausen & Field, 1996), and constitute universal function approximators, as powerful as networks with threshold or sigmoidal units (Maass, 1999; 2000).”
Synaptic Consolidation: From Synapses to Behavioral Modeling (2015, background): “…is represented by a set of highly active hippocampal neurons that form a memory engram (Ramirez et al., 2013) of the spatial context, similar to a k-winner-take-all system in artificial neural networks (Kohonen, 1982; Maass, 2000) or to sets of place cells in biology (O’Keefe and Nadal, 1971).”
Multi-class pattern classification using neural networks (2007, background): “Friedman showed that in some circumstances this algorithm is Bayes optimal [26].”
The atoms of neural computation (2014, methodology): “Maass (2000) studied the computational versatility of soft winner-take-all or soft-MAX operations, which were suggested by Douglas and Martin to be central to the operation of the putative canonical cortical microcircuit (Douglas & Martin, 2004) and which also feature heavily in a variety of other…”
On the computational power of circuits of spiking neurons (2004, background): “This assumption can in principle be satisfied by pools of spiking neurons since pools of spiking neurons can approximate any given continuous function on static inputs (see [1,16] for a proof).”
Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity (2013, background): “Such networks can be composed out of interconnected soft WTA circuits, which has been shown to be a powerful principle for designing neural networks that can solve arbitrary deterministic or stochastic computations [22,23,25].”