Pre-training of Mention Representations in Coreference Models

By Yuval Varkel, Amir Globerson. 2020

10 citations recorded by the source corpus.

Collecting labeled data for coreference resolution is a challenging task, requiring skilled annotators. It is thus desirable to develop coreference resolution models that can make use of unlabeled data. Here we provide such an approach for the powerful class of neural coreference models. These models rely on representations of mentions, and we show these representations can be learned in a self-supervised manner towards improving resolution accuracy. We propose two self-supervised tasks that are closely related to coreference resolution and thus improve mention representation. Applying this approach to the GAP dataset results in new state of the arts results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Information Extraction

Citations in the corpus

Learning to Retrieve Passages without Supervision (2021, neutral): “…spans for self-supervised pretraining has previously been considered for numerous tasks, e.g. coreference resolution and coref-erential reasoning (Kocijan et al., 2019; Varkel and Globerson, 2020; Ye et al., 2020) and question answering (Ram et al., 2021; Bian et al., 2021; Castel et al., 2021).”
Few-Shot Question Answering by Pretraining Span Selection (2021, methodology): “Similar ideas to recurring span selection were used for creating synthetic coreference resolution examples (Kocijan et al., 2019; Varkel and Globerson, 2020), which mask single words that occur multiple times in the same context.”