Higher-Order Coreference Resolution with Coarse-to-Fine Inference

By Kenton C. T. Lee, Luheng He, Luke S. Zettlemoyer. 2018

496 citations recorded by the source corpus.

We introduce a fully-differentiable approximation to higher-order inference for coreference resolution. Our approach uses the antecedent distribution from a span-ranking architecture as an attention mechanism to iteratively refine span representations. This enables the model to softly consider multiple hops in the predicted clusters. To alleviate the computational cost of this iterative process, we introduce a coarse-to-fine approach that incorporates a less accurate but more efficient bilinear factor, enabling more aggressive pruning without hurting accuracy. Compared to the existing state-of-the-art span-ranking approach, our model significantly improves accuracy on the English OntoNotes benchmark, while being far more computationally efficient.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Longformer: The Long-Document Transformer (2020, methodology): “…the complicated coarse-to-fine logic from Tensorflow in PyTorch (that involves a highly optimized custom GPU kernel originally released by Lee et al. (2018)), we devised a system where the lower transformer portion of the model passes activations and gradients back and forth between…”
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models (2021, methodology): “To expose this, Zhao et al. (2019) analyse the behaviour of a coreference resolution system proposed by Lee et al. (2018) with ELMo contextual weights on the WinoBias benchmark, revealing a signiﬁcant disparity in performance on the pro-and anti-stereotypical datasets.”
Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction (2022, background): “In practice, we could select some top-scoring quantities and build expressions on top of them (Lee et al., 2018).”
Gender Bias in Contextualized Word Embeddings (2019, methodology): “Previous work (Zhao et al., 2018a) evaluated the systems based on GloVe embeddings but here we evaluate a state-of-the-art system that trained on the OntoNotes corpus with ELMo embeddings (Lee et al., 2018).”
What do you learn from context? Probing for sentence structure in contextualized word representations (2019, background): “, 2017), ULMFit (Howard & Ruder, 2018), ELMo (Peters et al., 2018a; Lee et al., 2018; Strubell et al., 2018; Kitaev & Klein, 2018).”
Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, methodology): “When applied to a neural network based coreference resolution model (Lee et al., 2017, 2018) originally trained on OntoNotes 5.0 which was tested on WinoBias, gender augmentation lowered the difference be-tween F 1 scores on pro-stereotypical and anti-stereotypical test sets signiﬁcantly, which…”
Episodic Memory in Lifelong Language Learning (2019, background): “In language learning, progress in unsupervised pretraining (Peters et al., 2018; Howard & Ruder, 2018; Devlin et al., 2018) has driven advances in many language understanding tasks (Kitaev & Klein, 2018; Lee et al., 2018).”
Adapting Coreference Resolution for Processing Violent Death Narratives (2021, methodology): “Specifically, we apply the End-to-End Coreference Resolution (E2E-Coref) system (Lee et al., 2017, 2018), which has achieved high performance on the OntoNotes 5.0 (Hovy et al., 2006) corpus.”
Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation (2021, methodology): “SpanBERT introduces contextual span representation to the the e2e-coreference model (Lee et al., 2018; Joshi et al., 2019) to achieve state-of-the-art results on the English portion of the popular CoNLL-2012 shared task coreference benchmark (Pradhan et al., 2012).”
Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling (2019, methodology): “These were created by obtaining cluster predictions from the coreference model by Lee et al. (2018) and choosing a pronoun and the two candidate entities A and B from disjoint clusters.”
DeepStruct: Pretraining of Language Models for Structure Prediction (2022, methodology): “Comparison Methods We compare our method DEEPSTRUCT on the dataset to the following methods: (i) Higher-order c2f-coref (Lee et al., 2018): this method proposes a fully differentiable formulation of coreference resolution via iterative refinement using attention mechanism, (ii) BERT+c2f-coref…”
Finding a Balanced Degree of Automation for Summary Evaluation (2021, methodology): “For the Coreference Resolution model, we also use the out-of-the-box Coreference model pretrained by AllenNLP (Gardner et al., 2018).16 And it is based the model proposed by Lee et al. (2018).”
Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, background): “Sentence-level RE focuses on identifying relations be-tween entities within a single sentence.”
Pre-training of Mention Representations in Coreference Models (2020, methodology): “Almost all such models rely on vector representations of mentions (Clark and Manning, 2016; Lee et al., 2017, 2018; Denis and Baldridge, 2008; Rahman and Ng, 2009; Durrett et al., 2013; Chang et al., 2013; Wiseman et al., 2016; Martschat and Strube, 2015).”
Evaluation of named entity coreference (2019, methodology): “Notably, (Lee et al., 2018) misses less than 10% of the chains for all entity types compared to 20-40% by other systems.”
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (2018, methodology): “Our model draws from recent end-to-end span-based models for coreference resolution (Lee et al., 2017, 2018) and semantic role labeling (He et al., 2018) and extends them for the multi-task framework involving the three tasks of identiﬁcation of entity, relation and coreference.”
Where’s My Head? Definition, Data Set, and Models for Numeric Fused-Head Identification and Resolution (2019, methodology): “The work of (Peters et al., 2018; Lee et al., 2018; Zhang et al., 2018) further improve on their the scores with pre-training, refining span representation and using biaffine attention model for mention detection and clustering.”
BERT for Coreference Resolution: Baselines and Analysis (2019, methodology): “For our experiments, we use the higher-order coreference model in Lee et al. (2018) which is the current state of the art for the English OntoNotes dataset (Pradhan et al., 2012).”