BERT Masked Language Modeling for Co-reference Resolution

By Felipe Alfaro, Marta Ruiz Costa-jussà, José A. R. Fonollosa. 2019

11 citations recorded by the source corpus.

This paper explains the TALP-UPC participation for the Gendered Pronoun Resolution shared-task of the 1st ACL Workshop on Gender Bias for Natural Language Processing. We have implemented two models for mask language modeling using pre-trained BERT adjusted to work for a classification problem. The proposed solutions are based on the word probabilities of the original BERT model, but using common English names to replace the original test names.

Subfields: Large Language Models, Natural Language Processing, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

1. Marta Ruiz Costa-jussà
2. Will Radford
3. Kellie Webster
4. Christian Hardmeier

In Natural Language Processing

1. Kellie Webster
2. Marta Ruiz Costa-jussà
3. Christian Hardmeier
4. Will Radford
5. Hal Daumé Iii
6. Adina Williams
7. Liane Guillou
8. Yonatan Belinkov

In Information Extraction

1. Marta Ruiz Costa-jussà

Citations in the corpus

• Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019 (2019, methodology): “Two teams augmented the GAP data by using name substitutions (Liu, 2019; Lois et al., 2019) and two automatically created extra examples of the minority label Neither (Attree, 2019; Bao and Qiao, 2019).”