Fill the GAP: Exploiting BERT for Pronoun Resolution

By Kai-Chou Yang, Timothy Niven, Tzu-Hsuan Chou, Hung-Yu kao. 2019

4 citations recorded by the source corpus.

In this paper, we describe our entry in the gendered pronoun resolution competition which achieved fourth place without data augmentation. Our method is an ensemble system of BERTs which resolves co-reference in an interaction space. We report four insights from our work: BERT’s representations involve significant redundancy; modeling interaction effects similar to natural language inference models is useful for this task; there is an optimal BERT layer to extract representations for pronoun resolution; and the difference between the attention weights from the pronoun to the candidate entities was highly correlated with the correct label, with interesting implications for future work.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, methodology): “…here; most propose techniques we would have categorized as “Questionable correlations,” with a few as “Other representational harms” (Abzaliev, 2019; Attree, 2019; Bao and Qiao, 2019; Chada, 2019; Ionita et al., 2019; Liu, 2019; Lois et al., 2019; Wang, 2019; Xu and Yang, 2019; Yang et al., 2019).”
Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019 (2019, methodology): “An interesting ﬁnding was that certain BERT layers were more suitable for feature extraction than others (see Abza-liev (2019); Yang et al. (2019) for an exploration).”
Theories of “Gender” in NLP Bias Research (2022, unknown): “org/anthology/W19-3814/ [160] Kai-Chou Yang, Timothy Niven, Tzu-Hsuan Chou, and Hung-Yu Kao.”