Representation and structure in connectionist models

By Jeffrey L. Elman. 1991

Subfields: Representation & Self-Supervised Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Model Architectures

Citations in the corpus

Analysis Methods in Neural Language Processing: A Survey (2018, background): “While Elman’s work was limited in some ways, such as evaluating generalization or various linguistic phenomena—as Elman himself recognized (Elman, 1989)—it introduced methods that are still relevant today: from visualizing network activations in time, through clustering words by hidden state…”
Visualizing and Understanding Neural Models in NLP (2015, background): “Similarity is commonly visualized graphically, generally by projecting the embedding space into two dimensions and observing that similar words tend to be clustered together (e.g., Elman (1989), Ji and Eisenstein (2014), Faruqui and Dyer (2014)).”
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “Research on how to interpret and explain the decision process of Artiﬁcial Neural Networks (ANNs) has been going on since the late 1980’s (Elman, 1989; Andrews et al., 1995).”
Distributed representations, simple recurrent networks, and grammatical structure (1991, methodology): “In Elman (1990) a network similar to that in Figure 1 was trained to predict the order of words in simple (2- and 3-word) sentences.”
Visualizing Learning and Computation in Artificial Neural Networks (1992, methodology): “Hierarchical cluster analysis (Elman89; Sejnowski87), principal components analysis (Dennis91; Elman89), and canonical discriminant analysis (Dennis91; Wiles90) have been employed to gain insight into the classi cations made by ANNs.”
Connectionist sentence processing in perspective (1999, background): “A second source of difficulty in comparing connectionist and rule-based systems lies in the tendency of certain kinds of connectionist architecture to combine the roles of grammar and oracle or ambiguity-resolution device in a single representation distributed over a single set of hidden units.”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “Early work has clustered RNN activations showing that they organize in lexical categories [101, 102].”