Visualisation and 'Diagnostic Classifiers' Reveal how Recurrent and Recursive Neural Networks Process Hierarchical Structure (Extended Abstract)

By Dieuwke Hupkes, Willem H. Zuidema. 2018

25 citations recorded by the source corpus.

In this paper, we investigate how recurrent neural networks can learn and process languages with hierarchical, compositional semantics. To this end, we define the artificial task of processing nested arithmetic expressions, and study whether different types of neural networks can learn to compute their meaning. We find that simple recurrent networks cannot find a generalising solution to this task, but gated recurrent neural networks perform surprisingly well: networks learn to predict the outcome of the arithmetic expressions with high accuracy, although performance deteriorates somewhat with increasing length. We test multiple hypotheses on the information that is encoded and processed by the networks using a method called diagnostic classification. In this method, simple neural classifiers are used to test sequences of predictions about features of the hidden state representations at each time step. Our results indicate that the networks follow a strategy similar to our hypothesised ‘cumulative strategy’, which explains the high accuracy of the network on novel expressions, the generalisation to longer expressions than seen in training, and the mild deterioration with increasing length. This, in turn, shows that diagnostic classifiers can be a useful technique for opening up the black box of neural networks.

Subfields: Artificial Intelligence, Model Architectures, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Safety, Robustness & Interpretability

Citations in the corpus

Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure (2017, positive): “∗This paper is an extended abstract of a paper published in the Journal of Artificial Intelligence Research [Hupkes et al., 2018].”
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? (2021, background): “An alternate approach is probing (Adi et al., 2017; Conneau et al., 2018; Hupkes and Zuidema, 2018; Hewitt and Liang, 2019; Belinkov and Glass, 2019), which investigates how directly a model’s representations encode semantic relations by measuring if they can be easily decoded from them.”
Just Add Functions: A Neural-Symbolic Language Model (2019, background): “Other related work explores the use of language models to solve algebra word problems (Kushman et al. 2014; Roy and Roth 2015) or to evaluate nested arithmetic expressions (Hupkes and Zuidema 2018).”