Designing a Counter: Another Case Study of Dynamics and Activation Landscapes in Recurrent Networks

By Steffen Hölldobler, Yvonne Kalinke, Helko Lehmann. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets (2015, methodology): “This type of model has been widely used in the early nineties with some encouraging success (Das et al., 1992; 1993; Zeng et al., 1994; Holldobler et al., 1997; Mozer & Das, 1993).”
On Evaluating the Generalization of LSTM Models in Formal Languages (2018, background): “Similarly, H¨olldobler et al. (1997) demonstrated that a simple RNN architecture containing a single hidden unit with carefully tuned parameters can develop a canonical linear counting mechanism to recognize the simple context-free language a n b n , for n 250 .”
Computation in Recurrent Neural Networks: From Counters to Iterated Function Systems (1998, unknown): “3 This parameter transformation relates our results concerning the dynamics of iterated linear saturation functions to the results for iterated continuous sigmoidal functions described in [ 7 ].”
Incremental training of first order recurrent neural networks to predict a context-sensitive language (2003, unknown): “, 2000), context-free (Hölldobler et al., 1997; Finn, 1998) or contextsensitive (Steijvers and Grünwald, 1996).”
Synthetic Data for Deep Learning (2019, background): “For example, Joulin and Mikolov [294] present a new architecture (stack-augmented recurrent networks) to learn regularities in algorithmically generated sequences of symbols; the training data, as in previous such works [200,247,251,637], is synthetically generated by hand-crafted simple algorithms,”