A Recurrent Neural Network that Learns to Count

By P. Rodríguez. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets (2015, methodology): “Rodriguez et al. (1999) further studied the behavior of this network.”
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies (2016, neutral): “Simple recurrent networks struggled with this language (Rodriguez et al., 1999; Rodriguez, 2001).”
Deep learning in neural networks: An overview (2014, background): “LSTM also worked better on tasks involving context free languages (CFLs) that cannot be represented by HMMs or similar FSAs discussed in the RNN literature (Sun et al., 1993b; Wiles and Elman, 1995; Andrews et al., 1995; Steijvers and Grunwald, 1996; Tonkes and Wiles, 1997; Rodriguez et al., 1999; R”
Architectural Bias in Recurrent Neural Networks: Fractal Analysis (2002, background): “Results in this article extend the work of Blair, Bodén, Pollack, and others (Blair & Pollack, 1997; Bodén & Blair, 2002; Rodriguez et al., 1999) analyzing fixed-point representations of context-free languages in trained RNNs.”
Deep Recurrent Neural Networks for Hyperspectral Image Classification (2017, background): “An RNN [34], [35] is a class of artificial neural network that extends the conventional feedforward neural network with loops in connections.”
Fractal encoding of context‐free grammars in connectionist networks (2000, methodology): “: Rodriguez et al. (1999) Wiles and Elman (1995) study a backpropagation network that is trained on samples from the closely related language l n r n , n {1, 2, 3, % }.”
Context-free and context-sensitive dynamics in recurrent neural networks (2000, methodology): “The SRN has been shown to use qualitatively diﬀerent dynamics from the ﬁnite state automata emulations, which allow it to generalize when learning context-free languages (Wiles and Elman, 1995; Rodriguez and Wiles, 1998; Rodriguez et al., 1999; Bod´en et al., 1999).”
Deep Learning for Hyperspectral Image Classification: An Overview (2019, methodology): “RNNs were originally introduced in [39] and [40].”
Simple Recurrent Networks Learn Context-Free and Context-Sensitive Languages by Counting (2001, background): “Rodriguez, Wiles, and Elman (1999) showed that a simple recurrent network (SRN) can learn to process a simple context-free language (CFL) by counting up and down.”
Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets (2003, background): “Those that did not and those that even learned small CSL training sets (Rodriguez et al., 1999; Boden and Wiles, 2000) failed to extract the general rules and did not generalize well on substantially larger test sets.”
On Evaluating the Generalization of LSTM Models in Formal Languages (2018, methodology): “by the network, or the size of the largest accepted test set, or the accuracy on a fixed test set (Rodriguez et al., 1999; Bodén and Wiles, 2000; Gers and Schmidhuber, 2001; Rodriguez, 2001).”
Incremental training of first order recurrent neural networks to predict a context-sensitive language (2003, background): “A trajectory is said to be monotonic if its projections to the coordinate axes develop monotonically in time, otherwise it is called oscillating (Rodriguez et al., 1999).”
Rule Extraction from Recurrent Neural Networks: ATaxonomy and Review (2005, background): “A brief (and most probably inconclusive) list of examples of other analysis tools that have been used on RNNs is: Hinton diagrams (e.g. Hinton 1990, Niklasson & Bod´en 1997), Hierarchical Cluster Analysis (e.g. Cleeremans, McClelland & Servan-Schreiber 1989, Elman 1990, Servan-Schreiber, Cleeremans ”
Deep Learning for Anomaly Detection: A Survey (2019, background): “…network (DBN) (Salakhutdinov and Larochelle [2010]), generalized denoising autoencoders (Vincent et al. [2008]) , recurrent neural network (RNN) (Rodriguez et al. [1999]) Long short term memory networks (Lample et al. [2016]) which are used to detect outliers are discussed in detail in Section…”