Simple Recurrent Networks Learn Context-Free and Context-Sensitive Languages by Counting

By P. Rodríguez. 2001

102 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Electrical and Electronic Engineering, Model Architectures, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Electrical and Electronic Engineering

In Model Architectures

In Computational Linguistics & Semantics

Citations in the corpus

Neural methods for non-standard data (2004, background): “Further proto-typical training schemes and analysis have been presented e.g. in [78, 85].”
Visualisation and 'Diagnostic Classifiers' Reveal how Recurrent and Recursive Neural Networks Process Hierarchical Structure (Extended Abstract) (2018, unknown): “As we argue in the full paper, neither theoretical results [Siegelmann and Sontag, 1995], nor handcrafted examples [Gers and Schmidhuber, 2001; Rodriguez, 2001], nor large-scale NLP applications [Sutskever et al., 2014; Bahdanau et al., 2015; Kalchbrenner et al., 2014; Socher et al., 2013; Le and…”
On Evaluating the Generalization of LSTM Models in Formal Languages (2018, methodology): “…one may consider the length of the shortest sequence in a language whose output was incorrectly produced by the network, or the size of the largest accepted test set, or the accuracy on a ﬁxed test set (Ro-driguez et al., 1999; Bod´en and Wiles, 2000; Gers and Schmidhuber, 2001; Rodriguez, 2001).”
Transcoding Compositionally: Using Attention to Find More Generalizable Solutions (2019, background): “While (gated) recurrent neural networks are in principle capable of modeling compositional systems (e.g., Gers and Schmidhuber, 2001; Rodriguez, 2001), whether they in fact do so when trained on large amounts of data to perform natural language processing tasks remains an open question.”
An overview of reservoir computing: theory, applications and implementations (2007, background): “The literature describes many possible applications, including the learning of context free and context sensitive languages [41, 11], control and modeling of complex dynamical systems [54] and speech recognition [40, 12].”
Incremental training of first order recurrent neural networks to predict a context-sensitive language (2003, background): “All these studies including Rodriguez (2001) and Rodriguez et al. (1999) presented the strings in random order to the network. the epoch numbers of the set of solutions for each class of experiments.”
2007 Special Issue: An experimental unification of reservoir computing methods (2007, unknown): “Possible ‐ and actual ‐ applications are manifold and include the learning of context free and context sensitive languages (Gers & Schmidhuber, 2001; Rodriguez, 2001 ), control and modeling of complex dynamical systems (Suykens, Vandewalle, & De Moor, 1996) and speech recognition (Graves, Eck, Beri”
Diagnostic Classifiers Revealing how Neural Networks Process Hierarchical Structure (2016, background): “In the literature, one can ﬁnd many attempts at designing neural models that are able to capture hierarchical compositionality [e.g. Elman, 1990, Rodriguez, 2001, Martens, 2011], as well as a vast amount of papers on improved computational methods to train such models [e.g. Zeiler, 2012, Kingma and…”
Neural networks for relational learning: an experimental comparison (2011, background): “…al., 2001], theorem proving [Goller, 1997], image classification and object localization in images [Bianchini et al., 2003, Di Massa et al., 2006], language recognition [Rodriguez, 2001, Sturt et al., 2003], logo recognition [Francesconi et al., 1998] and web page ranking [Scarselli et al., 2005].”
The Acquisition of Anaphora by Simple Recurrent Networks (2013, methodology): “Rodríguez, Wiles & Elman (1999) and Rodríguez (2001) have shown that an SRN trained in this fashion is capable of inducing the kind of structurally based generalizations representable by the class of context-free grammars.”
On the Meaning of Words and Dinosaur Bones: Lexical Knowledge Without a Lexicon (2009, background): “This solution turns out to generalize to a number of more complex languages, including palindrome languages (e.g., characterized by sentences such as ACBDDBCA; Rodriguez, 2001; Rodriguez & Elman, 1999).”