Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

2021

Subfields: Natural Language Processing

Citations in the corpus

Few-shot Learning with Retrieval Augmented Language Models (2022, background): “for classification (Schick & Schütze, 2021a; Schick & Schutze, 2021; Gao et al., 2021; Tam et al., 2021) or generation (Schick & Schütze, 2021b).”
Improving Wikipedia verifiability with AI (2022, methodology): “The dense retrieval sub-system is a neural network which learns from Wikipedia data to encode the citation context into a dense query vector (Wu et al., 2019; Karpukhin et al., 2020; Maillard et al., 2021; Oğuz et al., 2021; Luan et al., 2021).”
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning (2021, background): “Efforts have also been made to tailor pre-training objectives to specific tasks, e.g., question answering (Ram et al., 2021; Jia et al., 2021), dialogue (Li et al., 2020), and span selection tasks (Joshi et al., 2020).”
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (2021, methodology): “We use sliding window inference as a tool to analyze our models, but note that it is normally prohibitively slow in practice (Press et al., 2021).”
Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022, background): “to minimize harms in downstream applications [12, 120], with international rights holders spanning multiple jurisdictions, and extend to multiple languages beyond the common English (further discussed by Blasi et al.”
Language Models are Multilingual Chain-of-Thought Reasoners (2022, background): “…help cross-lingual transfer on a wide range of NLP tasks such as cross-lingual named entity recognition (Pires et al., 2019; Mulcaire et al., 2019), zero-shot cross-lingual dependency parsing (Schuster et al., 2019; Shi et al., 2022b), and bilingual lexicon induction (Shi et al., 2021).”
TU Wien at TREC DL and Podcast 2021: Simple Compression for Dense Retrieval (2021, background): “[17] highlight the relation of linguistic style to peoples’ engagement with podcasts.”
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey (2021, methodology): “[202] utilize automatically produced syntax in both the pre-training and ine-tuning stages, and show improved performance on several benchmark datasets.”
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression (2023, background): “Prompt compression methods can be grouped into three main categories: (1) Token pruning (Goyal et al., 2020; Kim and Cho, 2021; Modar-ressi et al., 2022) and token merging (Bolya et al., 2023), which need model fine-tuning or intermediate results during inference and have been used with BERT-scale…”
Gender bias and stereotypes in Large Language Models (2023, background): “However, as is well known, language models perpetuate and occasionally amplify biases, stereotypes, and negative perceptions of minoritized groups in society [10, 13, 14, 66, 69, 84, 85, 90].”
Black-box Prompt Learning for Pre-trained Language Models (2022, background): “…(Wallace et al., 2019; Shin et al., 2020; Jiang et al., 2020; Yuan et al., 2021; Haviv et al., 2021; Gao et al., 2021; Ben-David et al., 2021) and continuous prompt (Zhong et al., 2021; Qin and Eisner, 2021; Hambardzumyan et al., 2021; Liu et al., 2021c; Han et al., 2021; Li and Liang, 2021).”
Biases in Large Language Models: Origins, Inventory, and Discussion (2023, result): “Pretrained language models are unmistakably and, sometimes, blatantly, biased in several respects, as numerous studies have shown over the years [1, 2, 9, 20, 66, 76, 93].”
Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits (2023, background): “Besides, speciﬁcally-designed prompts do not always exist in real-world human-AI interaction, and we cannot expect most users to know how to design appropriate prompts to improve the human-value alignment of an AI agent (Li & Liang, 2021).”
An Overview of Catastrophic AI Risks (2023, background): “racist and sexist views [24], and once those views are learned, it can be difficult to fully remove them.”
When and why vision-language models behave like bags-of-words, and what to do about it? (2022, background): “Order information in language and vision: Several works have highlighted the lack of word order sensitivity in large language models (Hessel & Schoﬁeld, 2021; O’Connor & Andreas, 2021; Pham et al., 2021; Sinha et al., 2021).”
What learning algorithm is in-context learning? Investigations with linear models (2022, methodology): “We can also cause K ( i ) to attend to an empty set by assuming the softmax has extra (“imaginary”) timestep obtained by prepending a 0 to attention vector pot-hoc (Chen et al., 2021).”