Edinburgh Research Explorer Understanding Learning Dynamics Of Language Models with SVCCA

119 citations recorded by the source corpus.

Research has shown that neural models implicitly encode linguistic features, but there has been no research showing how these encodings arise as the models are trained. We present the first study on the learning dynamics of neural language models, using a simple and flexible analysis method called Singular Vector Canonical Correlation Analysis (SVCCA), which enables us to compare learned representations across time and across models, without the need to evaluate directly on annotated data. We probe the evolution of syntactic, semantic, and topic representations, finding, for example, that part-of-speech is learned earlier than topic; that recurrent layers become more similar to those of a tagger during training; and embedding layers less similar. Our results and methods could inform better learning algorithms for NLP models, possibly to incorporate linguistic information more effectively.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives (2019, neutral): “Saphra and Lopez (2019) used SVCCA to investigate how representations of linguistic structure are learned over time in LMs.”
Designing and Interpreting Probes with Control Tasks (2019, background): “Previous work has made extensive use of control representations like non-contextual word embeddings or models with random weights (Belinkov et al., 2017; Tenney et al., 2019; Saphra and Lopez, 2019; Hewitt and Manning, 2019); our control tasks provide a complementary perspective, measuring a probe’s”
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML (2019, methodology): “Following several recent works [20, 25, 18, 16, 21, 8, 3] we measure this by applying Canonical Correlation Analysis (CCA) to the latent representations of the network.”
Information-Theoretic Probing with Minimum Description Length (2020, methodology): “We follow Tenney et al. (2019) and use ELMo (Peters et al.”
When Do You Need Billions of Words of Pretraining Data? (2020, background): “Other studies have investigate how one model’s linguistic knowledge changes during the training process, as a function of the number of updates (Saphra and Lopez, 2019; Chiang et al., 2020).”
Investigating Multilingual NMT Representations at Scale (2019, methodology): “, 2018), language models (Saphra and Lopez, 2018) and NMT (Bau et al.”
What Happens To BERT Embeddings During Fine-tuning? (2020, methodology): “Voita et al. (2019a) used a form of canonical correlation analysis (PW-CCA; Morcos et al., 2018) to study the layer-wise evolution of representations, while Saphra and Lopez (2019) explored how these representations evolve during training.”
Asking without Telling: Exploring Latent Ontologies in Contextual Representations (2020, methodology): “…which assess an encoder against particular data, descriptive methods analyze models on their own terms, and include clustering, visualization (Reif et al., 2019), and correlation analysis techniques (Voita et al., 2019; Saphra and Lopez, 2019; Abnar et al., 2019; Chrupała and Al-ishahi, 2019).”
Similarity Analysis of Contextual Word Representation Models (2020, methodology): “8 The fact that models within the same family are more similar to each other supports the choice of Saphra and Lopez (2019) to use models of similar architecture when probing models via similarity measures across tasks.”
Transfusion: Understanding Transfer Learning for Medical Imaging (2019, methodology): “A recent tool that effectively overcomes these challenges is (Singular Vector) Canonical Correlation Analysis, (SV)CCA [22, 19], which has been used to study latent representations through training, across different models, alternate training objectives, and other properties [22, 19, 25, 18, 8, 17, ”
Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model (2019, background): “SVCCA has been proposed to compare learned representations across language models (Saphra and Lopez, 2018).”
LINSPECTOR WEB: A Multilingual Probing Suite for Word Representations (2019, background): “This is considered a crucial feature as it provides insights on learning dynamics of models (Saphra and Lopez, 2019).”
Conditional probing: measuring usable information beyond a baseline (2021, background): “Other work performs structural analysis of representations without direct supervision (Saphra and Lopez, 2019; Wu et al., 2020).”
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (2022, background): “Previous works mostly focused on analyzing the evolution of hidden representations across layers (Voita et al., 2019), and probing the model with target tasks (Yang et al., 2020; Clark et al., 2019; Tenney et al., 2019; Saphra and Lopez, 2019).”
Is neural language acquisition similar to natural? A chronological probing study (2022, background): “The ﬁrst work on probing of neural networks across time was carried by (Saphra and Lopez, 2018).”
Transfusion: Understanding Transfer Learning with Applications to Medical Imaging (2019, unknown): “However, recent work Raghu et al. (2017); Saphra and Lopez (2018); Morcos et al. (2018); Magill et al. (2018) has shown the e ectiveness of CCA in performing representational comparisons across neural networks.”
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models (2022, unknown): “Saphra and Lopez [80] were the ﬁrst to analyze training dynamics for language modeling, focusing on the evolution of internal representations over pre-training.”