Easy Victories and Uphill Battles in Coreference Resolution

By Greg Durrett, Daniel Klein. 2013

241 citations recorded by the source corpus.

Classical coreference systems encode various syntactic, discourse, and semantic phenomena explicitly, using heterogenous features computed from hand-crafted heuristics. In contrast, we present a state-of-the-art coreference system that captures such phenomena implicitly, with a small number of homogeneous feature templates examining shallow properties of mentions. Surprisingly, our features are actually more effective than the corresponding hand-engineered ones at modeling these key linguistic phenomena, allowing us to win “easy victories” without crafted heuristics. These features are successful on syntax and discourse; however, they do not model semantic compatibility well, nor do we see gains from experiments with shallow semantic features from the literature, suggesting that this approach to semantics is an “uphill battle.” Nonetheless, our final system 1 outperforms the Stanford system (Lee et al. (2011), the winner of the CoNLL 2011 shared task) by 3.5% absolute on the CoNLL metric and outperforms the IMS system (Bj¨ orkelund and Farkas (2012), the best publicly available English coreference system) by 1.9% absolute.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

WinoGrande (2019, background): “ the system performance gap between “Gotcha” and “non-Gotcha” for each gender (lower the better). The ﬁrst three baselines are adopted from Rudinger et al. (2018); RULE is Lee et al. (2011), STATS is Durrett and Klein (2013), and NEURAL is Clark and Manning (2016). 5.3 Experimental Results Tables 6 ”
Gender Bias in Coreference Resolution (2018, methodology): “Statistical Statistical methods, often with millions of parameters, ultimately surpassed the performance of rule-based systems on shared task data (Durrett and Klein, 2013; Bj¨orkelund and Kuhn, 2014).”
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods (2018, methodology): “1 We use three different systems as prototypical examples: the Stanford Deterministic Coref-erence System (Raghunathan et al., 2010), the Berkeley Coreference Resolution System (Durrett and Klein, 2013) and the current best published system: the UW End-to-end Neural Coreference Resolution System (Le”
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing (2015, methodology): “Moreover, as shown in Table 4, mention clusters obtained by the transitive closure of the predicted pairwise coreference graph outperform both [36] and [37] when evaluated using standard Conll metrics on the quizbowl dataset.”
End-to-end Neural Coreference Resolution (2017, methodology): “, 2012; Björkelund and Kuhn, 2014; Martschat and Strube, 2015), or (4) mention-ranking models (Durrett and Klein, 2013; Wiseman et al., 2015; Clark and Manning, 2016a).”
Posterior calibration and exploratory analysis for natural language processing models (2015, methodology): “We use the Berkeley coreference resolution system (Durrett and Klein, 2013), which was originally presented as a CRF; we give it an equivalent a series of independent logistic regressions (see appendix for details).”
Deep Reinforcement Learning for Mention-Ranking Coreference Models (2016, methodology): “Although they are less expressive than entity-centric approaches to coreference (e.g., Haghighi and Klein, 2010), mention-ranking models are fast, scalable, and simple to train, causing them to be the dominant approach to coreference in recent years (Durrett and Klein, 2013; Wiseman et al., 2015).”
Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints (2016, background): “We focus on pronoun coreference because it is the most pressing manifestation of this problem and because existing coreference systems perform well on pronouns compared to harder instances of coreference (Durrett and Klein, 2013).”
Higher-Order Coreference Resolution with Coarse-to-Fine Inference (2018, background): “, 2017) that our proposed model builds upon, there is a large body of literature on coreference resolvers that fundamentally rely on scoring span pairs (Ng and Cardie, 2002; Bengtson and Roth, 2008; Denis and Baldridge, 2008; Fernandes et al., 2012; Durrett and Klein, 2013; Wiseman et al., 2015; Cla”
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks (2018, methodology): “Note that Durrett and Klein make use of external NLP tools including an automatic parser (Durrett and Klein 2013).”
A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, background): “Durrett and Klein [159] utilize antecedents representations to enable coreference inference through distribution features.”
BERT for Coreference Resolution: Baselines and Analysis (2019, background): “(2018) belongs to this family of models (Ng and Cardie, 2002; Bengtson and Roth, 2008; Denis and Baldridge, 2008; Fernandes et al., 2012; Durrett and Klein, 2013; Wiseman et al., 2015; Clark and Manning, 2016; Lee et al., 2017).”
Neural Probabilistic Model for Non-projective MST Parsing (2017, background): “, 2011), entity coreference resolution (Ng, 2010; Durrett and Klein, 2013; Ma et al., 2016), low-resource languages processing (McDonald et al.”
Neural Models for Reasoning over Multiple Mentions Using Coreference (2018, background): “tations from the Berkeley Entity Resolution system (Durrett and Klein, 2013) which gets about 62% F1 score on the CoNLL 2011 test set.”
Stack-Pointer Networks for Dependency Parsing (2018, background): “A wide range of NLP applications, such as coreference resolution (Ng, 2010; Durrett and Klein, 2013; Ma et al., 2016), sentiment analysis (Tai et al.”
Entity-Centric Coreference Resolution with Model Stacking (2015, background): “Recently, coreference models using latent antecedents have gained in popularity and achieved state-of-the-art results (Fernandes et al., 2012; Durrett and Klein, 2013; Chang et al., 2013; Björkelund and Kuhn, 2014).”
Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution (2015, methodology): “when predicting an antecedent for x, and another when predicting that x is non-anaphoric (Durrett and Klein, 2013).”