Improving Coreference Resolution by Learning Entity-Level Distributed Representations

By Kevin Clark, Christopher D. Manning. 2016

347 citations recorded by the source corpus.

A long-standing challenge in coreference resolution has been the incorporation of entity-level information - features defined over clusters of mentions instead of mention pairs. We present a neural network based coreference system that produces high-dimensional vector representations for pairs of coreference clusters. Using these representations, our system learns when combining clusters is desirable. We train the system with a learning-to-search algorithm that teaches it which local decisions (cluster merges) will lead to a high-scoring final coreference partition. The system substantially outperforms the current state-of-the-art on the English and Chinese portions of the CoNLL 2012 Shared Task dataset despite using few hand-engineered features.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Gender Bias in Coreference Resolution (2018, background): “Neural The move to deep neural models led to more powerful antecedent scoring functions, and the subsequent learned feature combinations resulted in new state-of-the-art performance (Wiseman et al., 2015; Clark and Manning, 2016b).”
Gender Bias in Neural Natural Language Processing (2018, methodology): “[2017], Clark and Manning [2016b], Zaremba et al. [2014] trained on benchmark datasets finds gender bias in these models 1. Next we turn our attention to mitigating the bias. Bolukbasi et al. [2016] introduced a technique for debiasing word embeddings which has been shown to mitigate unwanted associ”
Higher-Order Coreference Resolution with Coarse-to-Fine Inference (2018, background): “Recent coreference resolution systems have heavily relied on first order models (Clark and Manning, 2016a; Lee et al., 2017), where only pairs of entity mentions are scored by the model.”
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia (2018, background): “The performance drop of CorefNQG–mention-pair score shows the mention-pair score introduced from the external system (Clark and Manning, 2016) helps the neural network better encode coreference knowledge.”
Assessing The Factual Accuracy of Generated Text (2019, methodology): “tified with named-entity recognition (NER) systems like Chiu and Nichols [5], Finkel et al. [7], Lample et al. [9] that take in X and produce the set {ei}. NER is followed by co-reference resolution4 [6, 11, 24, 25]. Publicly available NER and co-reference systems include Stanford’s CoreNLP5 and NLT”
Dynamic Entity Representations in Neural Language Models (2017, background): “We suspect that a stronger baseline system (such as that of Clark and Manning, 2016a) could give greater improvements, if it can be adapted to provide k -best lists.”
Using Coreference Links to Improve Spanish-to-English Machine Translation (2017, background): “This is a naive representation, and more advanced work on entity-level representations has been performed in relation to coreference resolution (Clark and Manning, 2016; Wiseman et al., 2016), which could be applied here in the future.”
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (2018, background): “Among those studies, many papers assume the entity boundaries are given, such as (Clark and Manning, 2016), Adel and Sch¨utze (2017) and Peng et al. (2017).”
Paired Representation Learning for Event and Entity Coreference (2020, methodology): “For each topic in ECB, Cybulska and Vossen (2014) added a subtopic that talks about a different event belongs to a same topic.”
SmartPaste: Learning to Adapt Source Code (2017, positive): “Somewhat similar to our work, is the work of Clark and Manning [9], who create a neural model that learns to rank pairs of clusters of mentions to either merge them into a single co-reference entity or keep them apart.”
Evaluation of named entity coreference (2019, background): “…system, with its deterministic (Raghunathan et al., 2010; Lee et al., 2011; Recasens et al., 2013), statistical (Clark and Manning, 2015) and neural (Clark and Manning, 2016a,b) versions, and the neural end-to-end systems of (Lee et al., 2017) and (Lee et al., 2018) on traditional and NEC metrics.”
Deep Learning for Entity Matching: A Design Space Exploration (2018, methodology): “[10] used word embeddings to encode and rank pairs of entity mentions and use a deep neural network to identify clusters of entity mentions that correspond to the same entity, and Wiseman et al.”