Informing Unsupervised Pretraining with External Linguistic Knowledge

By Anne Lauscher, Ivan Vulic, Edoardo Maria Ponti, Anna Korhonen, Goran Glavas. 2019

41 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Xipeng Qiu
2. Jonas Pfeiffer
3. Iryna Gurevych
4. Tianxiang Sun
5. Nan Duan
6. Duyu Tang
7. Xuanjing Huang
8. Leonardo F. R. Ribeiro

Citations in the corpus

• K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2020, background): “LIBERT (Lauscher et al., 2019) fact triples from knowledge graph.”
• Pre-trained models for natural language processing: A survey (2020, background): “The external knowledge ranges from linguistic [94, 83, 136, 191], semantic [99], commonsense [52], factual [214, 136, 111, 202, 195], to domain-specific knowledge [58, 111].”
• Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers (2020, background): “…observation, most recent efforts focused on injecting factual (Zhang et al., 2019; Liu et al., 2019a; Peters et al., 2019) and linguistic knowledge (Lauscher et al., 2019; Peters et al., 2019) into pretrained LMs and demonstrated the usefulness of such knowledge in language understanding tasks…”