Transfer Learning from Pre-trained BERT for Pronoun Resolution
By Xingce Bao, Qianqian Qiao. 2019
17 citations recorded by the source corpus.
The paper describes the submission of the team “We used bert!” to the shared task Gendered Pronoun Resolution (Pair pronouns to their correct entities). Our final submission model based on the fine-tuned BERT (Bidirectional Encoder Representations from Transformers) ranks 14th among 838 teams with a multi-class logarithmic loss of 0.208. In this work, contribution of transfer learning technique to pronoun resolution systems is investigated and the gender bias contained in classification models is evaluated.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hal Daumé Iii
- Marta Ruiz Costa-jussà
- Su Lin Blodgett
- Lawrence Carin
- Kellie Webster
- Hanna M. Wallach
- Solon Barocas
- Yejin Choi
In Natural Language Processing
- Hal Daumé Iii
- Marta Ruiz Costa-jussà
- Su Lin Blodgett
- Kellie Webster
- Lawrence Carin
- Christian Hardmeier
- Will Radford
- Hanna M. Wallach
Citations in the corpus
- Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, methodology): “…here; most propose techniques we would have categorized as “Questionable correlations,” with a few as “Other representational harms” (Abzaliev, 2019; Attree, 2019; Bao and Qiao, 2019; Chada, 2019; Ionita et al., 2019; Liu, 2019; Lois et al., 2019; Wang, 2019; Xu and Yang, 2019; Yang et al., 2019).”
- Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019 (2019, background): “Two teams augmented the GAP data by using name substitutions (Liu, 2019; Lois et al., 2019) and two automatically created extra examples of the minority label Neither (Attree, 2019; Bao and Qiao, 2019).”
- On Leveraging Pretrained GANs for Generation with Limited Data (2020, methodology): “…of a pretrained classifier) (Bengio, 2012; Donahue et al., 2014; Luo et al., 2017; Zamir et al., 2018), and transfer learning in natural language processing (e.g., to reuse the expensively-pretrained BERT model) (Devlin et al., 2018; Bao & Qiao, 2019; Peng et al., 2019; Mozafari et al., 2019).”
- On Leveraging Pretrained GANs for Limited-Data Generation (2020, background): “…extractor of a pretrained classifier) (Bengio, 2012; Donahue et al., 2014; Luo et al., 2017; Zamir et al., 2018), and transfer learning in natural language processing (e.g., to reuse the expensively-pretrained BERT (Devlin et al., 2018)) (Bao & Qiao, 2019; Peng et al., 2019; Mozafari et al., 2019).”