WikiCREM: A Large Unsupervised Corpus for Coreference Resolution

By Vid Kocijan, Oana-Maria Camburu, Ana-Maria Cretu, Yordan Yordanov, Phil Blunsom, Thomas Lukasiewicz. 2019

30 citations recorded by the source corpus.

Pronoun resolution is a major area of natural language understanding. However, large-scale training sets are still scarce, since manually labelling data is costly. In this work, we introduce WikiCREM (Wikipedia CoREferences Masked) a large-scale, yet accurate dataset of pronoun disambiguation instances. We use a language-model-based approach for pronoun resolution in combination with our WikiCREM dataset. We compare a series of models on a collection of diverse and challenging coreference resolution problems, where we match or outperform previous state-of-the-art approaches on 6 out of 7 datasets, such as GAP, DPR, WNLI, PDP, WinoBias, and WinoGender. We release our model to be used off-the-shelf for solving pronoun disambiguation.

Subfields: Artificial Intelligence, Natural Language Processing, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Maosong Sun
3. Jonathan Berant
4. Caiming Xiong
5. Mohit Bansal
6. Percy Liang
7. Hannaneh Hajishirzi
8. Omer Levy

In Natural Language Processing

1. Zhiyuan Liu
2. Maosong Sun
3. Caiming Xiong
4. Jonathan Berant
5. Mohit Bansal
6. Hannaneh Hajishirzi
7. Amir Globerson
8. Omer Levy

In Information Extraction

1. Omer Levy
2. Jonathan Berant
3. Ori Ram
4. Amir Globerson
5. Maosong Sun
6. Caiming Xiong
7. Yankai Lin
8. Zhiyuan Liu

Citations in the corpus

• Few-Shot Question Answering by Pretraining Span Selection (2021, methodology): “Taking advantage of recurring words in a passage (restricted to nouns or named entities) was proposed in past work as a signal for coreference (Kocijan et al., 2019; Ye et al., 2020).”
• Pre-training of Mention Representations in Coreference Models (2020, background): “Kocijan et al. (2019) finds pairs of sentences with at least two distinct”
• Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting (2021, methodology): “In To find the reference of each pronoun in the template sentences, we follow (Kocijan et al., 2019) approach.”
• Coreferential Reasoning Learning for Language Representation (2020, background): “Benefiting from the augmented data, Kocijan et al. (2019a) achieves state-of-the-art in sentence-level coreference resolution.”