Gendered Pronoun Resolution using BERT and an Extractive Question Answering Formulation

By Rakesh Chada. 2019

10 citations recorded by the source corpus.

The resolution of ambiguous pronouns is a longstanding challenge in Natural Language Understanding. Recent studies have suggested gender bias among state-of-the-art coreference resolution systems. As an example, Google AI Language team recently released a gender-balanced dataset and showed that performance of these coreference resolvers is significantly limited on the dataset. In this paper, we propose an extractive question answering (QA) formulation of pronoun resolution task that overcomes this limitation and shows much lower gender bias (0.99) on their dataset. This system uses fine-tuned representations from the pre-trained BERT model and outperforms the existing baseline by a significant margin (22.2% absolute improvement in F1 score) without using any hand-engineered features. This QA framework is equally performant even without the knowledge of the candidate antecedents of the pronoun. An ensemble of QA and BERT-based multiple choice and sequence classification models further improves the F1 (23.3% absolute improvement upon the baseline). This ensemble model was submitted to the shared task for the 1st ACL workshop on Gender Bias for Natural Language Processing. It ranked 9th on the final official leaderboard.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, methodology): “…here; most propose techniques we would have categorized as “Questionable correlations,” with a few as “Other representational harms” (Abzaliev, 2019; Attree, 2019; Bao and Qiao, 2019; Chada, 2019; Ionita et al., 2019; Liu, 2019; Lois et al., 2019; Wang, 2019; Xu and Yang, 2019; Yang et al., 2019).”
Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019 (2019, methodology): “Two other systems stood out as novel in their approach to the task: Chada (2019) reformulated GAP reference resolution as a question answering task, and Lois et al. (2019) used BERT in a third way, directly applying the masked language modeling task to predicting resolutions.”
Pre-training of Mention Representations in Coreference Models (2020, result): “There are however previous works (Attree, 2019; Chada, 2019) that consider the gold-two-mention task (Webster et al., 2018), where the locations of the gold names and pronoun are used during inference as well 1 .”