Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks

By Nils Bertschinger, Thomas Natschläger. Neural Computation, 2004

841 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/04c6461e524107198eb98835cd6ba2f7733b3163

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

Re-visiting the echo state property (2012, background): “This line of thought apparently originates – within the reservoir computing arena – in [2, 1], where it was shown that, for reservoirs with binary threshold units, performance peaked when the reservoir weights were scaled to just below a critical value after which state convergence was lost.”
Self-organized criticality as a fundamental property of neural systems (2014, background): “(Kauffman, 1984; Derrida and Pomeau, 1986), liquid state machines (Langton, 1990), and neuronal networks (Maass et al., 2002; Bertschinger and Natschläger, 2004), for a review also see Legenstein and Maass (2007).”
Mean Field Residual Networks: On the Edge of Chaos (2017, background): “It has been argued in many prior works [1, 9] that effective computation can only be supported by a dynamical behavior that is on the edge of chaos.”
Generating Coherent Patterns of Activity from Chaotic Neural Networks (2009, neutral): “Such an input can induce a transition between chaotic and nonchaotic states (Molgedey et al., 1992; Bertschinger and Natschlä ger, 2004; K. Rajan, L.F.A., and H. Sompolinsky, unpublished data).”
Encoding and decoding of overlapping odor sequences. (2006, positive): “It was suggested that, to perform complex computations on time-varying input, such networks need to operate at the edge of dynamical regimes where behavior becomes chaotic (Bertschinger and Natschlager, 2004).”
Optimal Hierarchical Modular Topologies for Producing Limited Sustained Activation of Neural Networks (2009, positive): “While its functional signiﬁ cance is still not well understood, it has been suggested that critical dynamics may enhance information processing capabilities of neuronal networks (e.g., Bertschinger and Natschläger, 2004).”
2007 Special Issue: Edge of chaos and prediction of computational performance for neural circuit models (2007, neutral): “Networks of threshold elements with online input were recently considered in Bertschinger and Natschläger (2004). In this study, a network with online input is called chaotic if arbitrary small differences in a (initial) network state x(0) are highly amplified and do not vanish.”
Colloquium: Criticality and dynamical scaling in living systems (2017, background): “…such machines can perform real-time computations –responding rapidly to time varying input signals– in a coherent yet flexible way if they operate near a critical point (Bertschinger and Natschlager, 2004; Boedecker et al., 2012; Legenstein and Maass, 2007; Legenstein, 2005; Maass et al., 2002).”
Griffiths phases and the stretching of criticality in brain networks (2013, background): “Critical dynamics provides a delicate trade-off between these two impractical tendencies, and has been argued to imply optimal transmission and storage of information [7, 8, 17], optimal computational capabilities [18], large network stability [16], maximal variety of memory repertoires [19], and ma”
SORN: A Self-Organizing Recurrent Neural Network (2009, positive): “It has been argued that a tuning of network dynamics to criticality should bring the performance of static reservoir networks close to the optimal performance (Bertschinger and Natschläger, 2004).”
Neuronal Avalanches Imply Maximum Dynamic Range in Cortical Networks at Criticality (2009, background): “…map input differences into distinguishable network outputs, our result is also closely related to network-mediated separation, which has been predicted to peak in the critical state, at the transition from ordered to chaotic dynamics (Bertschinger and Natschlager, 2004;Legenstein and Maass, 2007).”
Randomness in neural networks: an overview (2017, background): “One important aspect that we briefly mention here is the concept of ‘edge of instability’ (or ‘edge of chaos’); namely, a large body of literature suggests that ESNs, like other recurrent architectures and even cortical circuits, have the highest computational power when they operate near an instabl”