Learning Global Features for Coreference Resolution

By Sam Wiseman, Alexander M. Rush, Stuart M. Shieber. 2016

200 citations recorded by the source corpus.

There is compelling evidence that coreference prediction would benefit from modeling global information about entity-clusters. Yet, state-of-the-art performance can be achieved with systems treating each mention prediction independently, which we attribute to the inherent difficulty of crafting informative cluster-level features. We instead propose to use recurrent neural networks (RNNs) to learn latent, global representations of entity clusters directly from their mentions. We show that such representations are especially useful for the prediction of pronominal mentions, and can be incorporated into an end-to-end coreference system that outperforms the state of the art without requiring any additional search.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Gender Bias in Coreference Resolution (2018, methodology): “Global inference over these models further improved performance (Wiseman et al., 2016; Clark and Manning, 2016a), but from the perspective of potential bias, the information available to the model is largely the same as in the statistical models.”
End-to-end Neural Coreference Resolution (2017, background): “All recent coreference models, including neural approaches that achieved impressive performance gains (Wiseman et al., 2016; Clark and Manning, 2016b,a), rely on syntactic parsers, both for head-word features and as the input to carefully hand-engineered mention proposal algorithms.”
Deep Reinforcement Learning for Mention-Ranking Coreference Models (2016, background): “…are typically trained with heuristic loss functions that assign costs to different error types, as in the heuristic loss we describe in Section 3.1 (Fernandes et al., 2012; Durrett et al., 2013; Bj¨orkelund and Kuhn, 2014; Wiseman et al., 2015; Martschat and Strube, 2015; Wiseman et al., 2016).”
Reference-Aware Language Models (2016, background): “There are also lots of works on coreference resolution (Haghighi and Klein, 2010; Wiseman et al., 2016).”
Dynamic Entity Representations in Neural Language Models (2017, background): “In previous work, such information has been added as features (Luo et al., 2004; Bj¨orkelund and Kuhn, 2014) or by computing distributed entity representations (Wiseman et al., 2016; Clark and Manning, 2016b).”
Context-Aware Neural Machine Translation Learns Anaphora Resolution (2018, background): “One option would be to integrate ‘global’ features summarizing properties of groups of mentions predicted as linked in a document (Wiseman et al., 2016), or to use latent relations to trace entities across documents (Ji et al., 2017).”
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (2018, methodology): “…modeling entities, relations, and coreference links, most recent neural models for these tasks focus on single tasks (Clark and Manning, 2016; Wiseman et al., 2016; Lee et al., 2017; Lam-ple et al., 2016; Peng et al., 2017) or joint entity and relation extraction (Katiyar and Cardie, 2017;…”
The KnowRef Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution (2018, background): “Later, lexical learning approaches emerged as the new state of the art (Durrett and Klein, 2013), followed more recently by neural models (Wiseman et al., 2016; Clark and Manning, 2016).”
Neural Models for Reasoning over Multiple Mentions Using Coreference (2018, background): “At the same time, systems for coreference resolution have seen a gradual increase in accuracy over the years (Durrett and Klein, 2013; Wiseman et al., 2016; Lee et al., 2017).”
Syntactic Scaffolds for Semantic Structures (2018, background): “…syntax as T2 in a pipeline is perhaps the most common approach for semantic structure prediction (Toutanova et al., 2008; Yang and Mitchell, 2017; Wiseman et al., 2016).2 However, pipelines introduce the problem of cascading errors (T2’s mistakes affect the performance, and perhaps the training,…”
Resolving Gendered Ambiguous Pronouns with BERT (2019, unknown): “They run 4 state-of-the-art coreference resolution models (Lee et al., 2013; Clark and Manning, 2015; Wiseman et al., 2016; Lee et al., 2017) on the OntoNotes and GAP datasets reporting F1 scores separately for masculine and feminine pronoun-named entity pairs (metrics M and F in the paper).”
Look Again at the Syntax: Relational Graph Convolutional Network for Gendered Ambiguous Pronoun Resolution (2019, methodology): “…methods: (1) Off-the-shelf resolvers including a rule-based system of Lee et al. (2013) and three neural resolvers from Clark and Manning (2015), Wiseman et al. (2016), and Lee et al. (2017); (2) Baselines based on traditional cues for coreference; (3)Baselines based on structural cues:…”
CorefQA: Coreference Resolution as Query-based Span Prediction (2019, methodology): “Earlier neural-based models (Wiseman et al., 2016; Clark and Manning, 2015, 2016) rely on parsers and hand-engineered mention proposal algorithms.”
Pre-training of Mention Representations in Coreference Models (2020, background): “Almost all such models rely on vector representations of mentions (Clark and Manning, 2016; Lee et al., 2017, 2018; Denis and Baldridge, 2008; Rahman and Ng, 2009; Durrett et al., 2013; Chang et al., 2013; Wiseman et al., 2016; Martschat and Strube, 2015).”
Robustness in Coreference Resolution (2020, methodology): “For the mention-ranking model, Clark & Manning (2016a) use the training objective proposed by Wiseman et al. (2015) that encourages separating the highest scoring true antecedent and incorrect antecedents. The loss function of mention-ranking introduced by Wiseman et al. (2015) is as follows:”