Representation of Linguistic Form and Function in Recurrent Neural Networks

By Ákos Kádár, Grzegorz Chrupała, Afra Alishahi. 2016

164 citations recorded by the source corpus.

We present novel methods for analyzing the activation patterns of recurrent neural networks from a linguistic point of view and explore the types of linguistic structure they learn. As a case study, we use a standard standalone language model, and a multi-task gated recurrent network architecture consisting of two parallel pathways with shared word embeddings: The Visual pathway is trained on predicting the representations of the visual scene corresponding to an input sentence, and the Textual pathway is trained to predict the next word in the same sentence. We propose a method for estimating the amount of contribution of individual tokens in the input to the final prediction of the networks. Using this method, we show that the Visual pathway pays selective attention to lexical categories and grammatical functions that carry semantic information, and learns to treat word types differently depending on their grammatical function and their position in the sequential structure of the sentence. In contrast, the language models are comparatively more sensitive to words with a syntactic function. Further analysis of the most informative n-gram contexts for each model shows that in comparison with the Visual pathway, the language models react more strongly to abstract contexts that represent syntactic constructions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Dissecting Contextual Word Embeddings: Architecture and Representation (2018, background): “K´ad´ar et al. (2017) attributed the activation patters of RNNs to input tokens and showed that a RNN language model is strongly sensitive to tokens with syntactic functions.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “2015; Kádár et al., 2017; Qian et al., 2016a; Liu et al., 2018) and speech (Wu and King, 2016; Nagamine et al., 2015; Wang et al., 2017b).”
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (2018, methodology): “…of recurrent networks and related models by using visualization, input/hidden representation deletion techniques or by looking at the word-by-word behaviour of the network (e.g., Nagamine et al., 2015; Hupkes et al., 2017; Li et al., 2016; Linzen et al., 2016; Kàdàr et al., 2017; Li et al., 2017).”
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies (2016, neutral): “K´ad´ar et al. (2016) and Li et al. (2016) suggest visualization techniques for word-level RNNs trained to perform tasks that aren’t explicitly syntactic (image captioning and sentiment analysis).”
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information (2018, methodology): “Popular approaches include visualising the state space of these networks, performing ablations to the network, or using the internal states of the networks for some auxiliary task (e.g., Adi et al., 2016; Kádár et al., 2017; Conneau et al., 2018; Khandelwal et al., 2018).”
Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks (2017, methodology): “Techniques for analyzing neural network models include visualization of hidden units (Elman, 1991; Karpathy et al., 2015; K´ad´ar et al., 2016; Qian et al., 2016a), which provide illuminating, but often anecdotal information on how the network works.”
Understanding Neural Networks through Representation Erasure (2016, methodology): “Kádár et al. (2016) propose methods for analyzing the activation patterns of RNNs from a linguistic point of view.”
Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task (2017, background): “Additionally, there is an increasing amount of work on developing methods of exposing exactly what linguistic knowledge NLP models develop (Kádár et al., 2016; Li et al., 2015) and what linguistic information is encoded in models’ produced representations (Adi et al., 2016; Ettinger et al., 2016).”
Assessing Composition in Sentence Vector Representations (2018, methodology): “…to increase interpretability of neural network models in NLP—including use of visualization to analyze what neural networks learn (Li et al., 2015; K´ad´ar et al., 2016), efforts to increase interpretability by generating explanations of model predictions (Ribeiro et al., 2016; Lei et al., 2016;…”
Identifying and Controlling Important Neurons in Neural Machine Translation (2018, positive): “Similar to previous work analyzing neural networks in NLP (Elman, 1991; Karpathy et al., 2015; K ´ ad ´ ar et al., 2016), we visualize activations of neurons and observe interpretable behavior.”
Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks (2016, background): “In [10,11], the authors show that RNNs speciﬁcally learn lexical categories and grammatical functions that carry semantic information, partially by modifying the inputs fed to the model.”
Encoding of phonology in a recurrent neural model of grounded speech (2017, background): “There are a small number of studies which focus on the syntax and/or semantics in the context of neural models of written language (e.g. Elman, 1991; Frank et al., 2013; Kádár et al., 2016; Li et al., 2016a; Adi et al., 2016; Li et al., 2016b; Linzen et al., 2016).”
Techniques for interpretable machine learning (2018, methodology): “res such as quotation marks, brackets, and line lengths in a text. In another work, a wordlevel language model is utilized to analyze the linguistic features encoded by individual hidden units of RNN [17]. The visualization results illustrate that some dimensions of units are mostly activated by cer”
RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency (2018, background): “…al., 2016; Gulordava et al., 2018; Wilcox et al., 2018) and complements a variety of other approaches currently practiced (Shi et al., 2016; Belinkov et al., 2018; Blevins et al., 2018; K´ad´ar et al., 2017; Williams et al., 2018; Et-tinger et al., 2017; Lake and Baroni, 2017; Weber et al., 2018).”
RNNs Implicitly Implement Tensor Product Representations (2018, unknown): “Another approach is based on probing tasks, which assess what information can be easily decoded from a vector representation (Shi et al. 2016; Belinkov et al. 2017; K´ad´ar et al. 2017; Ettinger et al. 2018; compare work in cognitive neuro-science, e.g. Norman et al. 2006).”