Correlating Neural and Symbolic Representations of Language

By Grzegorz Chrupała, Afra Alishahi. 2019

75 citations recorded by the source corpus.

Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant approach in NLP. Here we present two methods based on Representational Similarity Analysis (RSA) and Tree Kernels (TK) which allow us to directly quantify how strongly the information encoded in neural activation patterns corresponds to information represented by symbolic structures such as syntax trees. We first validate our methods on the case of a simple synthetic language for arithmetic expressions with clearly defined syntax and semantics, and show that they exhibit the expected pattern of results. We then our methods to correlate neural representations of English sentences with their constituency parse trees.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Knowledge Graphs & Representation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Knowledge Graphs & Representation

Citations in the corpus

Probing Classifiers: Promises, Shortcomings, and Advances (2021, methodology): “2 We use fl(x) to refer more generally to any intermediate output of f when applied to x, so the framework includes analyses of other model components, such as attention weights (Clark et al. 2019). 3 For chronological completeness, workshop and preprint versions of Hupkes, Veldhoen, and Zuidema (20”
BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance (2019, background): “Many recent papers have sought a deeper understanding of BERT, whether to assess its encoding of sentence structure (Lin et al., 2019; Hewitt and Manning, 2019; Chrupała and Alishahi, 2019; Jawahar et al., 2019; Tenney et al., 2019b); its representational structure more generally (Abnar et al.”
Probing Natural Language Inference Models through Semantic Fragments (2019, background): “ludes work on syntactic probing (McCoy, Pavlick, and Linzen 2019; Goldberg 2019), probing basic reasoning (Weston et al. 2015; Geiger et al. 2018; 2019) and probing other tasks (Lake and Baroni 2017; Chrupała and Alishahi 2019; Warstadt et al. 2019). Geiger et al. (2018) is the closest work to ours.”
Predicting Inductive Biases of Pre-Trained Models (2021, methodology): “Similar techniques include attention maps/visualizations (Voita et al., 2019; Serrano & Smith, 2019), and relational similarity analyses (Chrupała & Alishahi, 2019).”
UnNatural Language Inference (2020, background): “, 2016) that transformerbased models pretrained on LM do acquire some knowledge of natural language syntax (Gulordava et al., 2018; Chrupała and Alishahi, 2019; Jawahar et al., 2019; Lin et al., 2019; Manning et al., 2020; Hawkins et al., 2020; Linzen and Baroni, 2021).”
Analyzing Redundancy in Pretrained Transformer Models (2020, methodology): “The general importance of neurons is mainly captured using similarity and correlationbased methods (Raghu et al., 2017; Chrupała and Alishahi, 2019; Wu et al., 2020).”
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge (2019, methodology): “Our assumption, similar to Clark et al. (2020), is that the ability of a model to mimic the input-output behavior of data generated using expert knowledge gives some evidence of correctness in virtue of such data being correct by construction (see discussion by Ranta (2017)).”
Blackbox Meets Blackbox: Representational Similarity & Stability Analysis of Neural Language Models and Brains (2019, background): “RSA can also be applied to deep learning models (Laakso and Cottrell, 2000; Dharmaretnam and Fyshe, 2018; Alvarez-Melis and Jaakkola, 2018; Wang et al., 2018; Chrupała and Alishahi, 2019).”
Probing Classifiers: Promises, Shortcomings, and Alternatives (2021, background): “…Mur, and Bandettini 2008) to measure similarity between word or sentence representations and syntactic properties, both local properties like determining a verb’s subject (Lepori and McCoy 2020) and more structured properties like inferring the full syntactic tree (Chrupała and Alishahi 2019).”
Similarity Analysis of Contextual Word Representation Models (2020, methodology): “has been used to analyze neural network representations (Bouchacourt and Baroni, 2018; Chrupała and Alishahi, 2019; Chrupała, 2019), or other variants of CCA, such as deep CCA (Andrew et al.”
What Happens To BERT Embeddings During Fine-tuning? (2020, unknown): “, 2008), it has become increasingly used to analyze similarity between neural network activations (Abnar et al., 2019; Chrupała and Alishahi, 2019).”
All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality (2021, background): “, 2019), syntax structure (Chrupała and Alishahi, 2019), semantic shift (Martinc et al.”
Analyzing analytical methods: The case of phonology in neural models of spoken language (2020, methodology): “Previous work has shown that sometimes non-trivial representations can be found in the activation patterns of randomly initialized, untrained neural networks (Zhang and Bowman, 2018; Chrupała and Alishahi, 2019).”
Picking BERT’s Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis (2020, unknown): “epresentations of those same words in the text or visual domain (Chrupała, 2019), and to compare the representational spaces of two agents in an emergent language game (Bouchacourt and Baroni, 2018). Chrupała and Alishahi (2019) also employ RSA to study the correspondence between the representationa”
On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation (2021, methodology): “RSA has been widely used to analyze the similarity between two neural network outputs (Abnar et al., 2019; Chrupała and Alishahi, 2019; Merchant et al., 2020), which works by creating two comparable sets of representations by inputting a same set of n samples to the two models.”
Interpretability and Analysis in Neural NLP (2020, background): “, 2019; Ross and Pavlick, 2019), psycholinguistic methods (Ettinger et al., 2018; Chrupała and Alishahi, 2019), layerwise analyses (Peters et al.”
Are Multilingual Neural Machine Translation Models Better at Capturing Linguistic Features? (2020, background): “Chrupała and Alishahi (2019) use representational similarity analysis to compare the metrics induced by sentence representations and syntactic dependency trees.”