BERT Rediscovers the Classical NLP Pipeline

By Ian Tenney, Dipanjan Das, Ellie Pavlick. 2019

1,962 citations recorded by the source corpus.

Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captured within the network. We find that the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way, and that the regions responsible for each step appear in the expected sequence: POS tagging, parsing, NER, semantic roles, then coreference. Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically, revising lower-level decisions on the basis of disambiguating information from higher-level representations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Experience Grounds Language (2020, background): “tation can discover both rich syntax and semantics without our help (Tenney et al., 2019).”
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, background): “…where h̄(x) is set as the average of the last two embedding vectors h(L−1)(x) and h(L)(x) based on the following two reasons: First, we want to capture high-level semantics (e.g., sentiments) and embedding in later layers represents higher level semantics (Tenney et al., 2019).”
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives (2019, background): “This agrees with the work by Tenney et al. (2019a).”
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (2018, background): “Random Encoder A number of recent works have noted that randomly initialized, untrained LSTMs can obtain surprisingly strong downstream task performance (Zhang and Bowman, 2018; Wieting and Kiela, 2019; Tenney et al., 2019b).”
HuggingFace's Transformers: State-of-the-art Natural Language Processing (2019, neutral): “Some examples include Tenney et al. (2019), Michel et al. (2019), Clark et al. (2019b).”
oLMpics-On What Language Model Pre-training Captures (2019, background): “…that these LM compute, revealing that they encode substantial amounts of syntax and semantics (Linzen et al., 2016b; Peters et al., 2018b; Tenney et al., 2019b; Goldberg, 2019; Hewitt and Manning, 2019; Ten-ney et al., 2019a; Shwartz and Dagan, 2019; Lin et al., 2019; Coenen et al.,…”
Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “Analysis of NLP Models Structural tests for analyzing models’ internals include probing (Tenney et al., 2019) and attention analysis (Jain and Wal-lace, 2019; Serrano and Smith, 2019; Wiegreffe and Pinter, 2019; Tutek and Snajder, 2020).”
Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving (2019, methodology): “This was shown for BERT’s hidden activation states in (Hewitt & Manning, 2019; Tenney et al., 2019) and, most directly related to the present work, for the graph implicit in BERT’s attention weights (Coenen et al.”
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022, background): “Additionally, we encourage further investigations into probing the latent knowledge within LLMs (Tenney et al., 2019; Li et al., 2021) or examining how LLMs handle entities and people (Onoe et al.”
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers (2022, methodology): “At high level, transformer models can be broadly categorized to three branches: encoder-only models (BERT) [63], decoder-only models (GPT-3-style) [48], and encoder-decoder models (T5) [49].”
HuggingFace's Transformers: State-of-the-art Natural Language Processing (2019, neutral): “They used the Transformers API as a generic front-end and performed fine-tuning on a variety of different models, leading to research on the structure of BERT (Tenney et al., 2019).”
Unsupervised Speech Recognition (2021, background): “Similarly in natural language understanding, probing Transformer layers of the BERT models showed that blocks behave differently, with earlier blocks encoding syntactic information, while high-level semantic information appears at higher blocks (Tenney et al., 2019; Jawahar et al., 2019).”
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models (2021, background): “This is in line with the probing results of Hewitt and Manning (2019) and Tenney et al. (2019a), who find that the highest amount of syntactic information is encoded in the uppermiddle layers.”
Measuring and Improving Consistency in Pretrained Language Models (2021, background): “Other work aims to quantify the degree to which KBs are inconsistent and detects inconsistent statements (Thimm, 2009, 2013; Muiño, 2011).”
Of Non-Linearity and Commutativity in BERT (2021, methodology): “The authors of [31] and [13] show that features are extracted in BERT following the same order as in the classical NLP pipeline, with syntactic features extracted in earlier layers and semantic features in later layers.”