DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference

By Shikhar Murty, Tatsunori B. Hashimoto, Christopher D. Manning. 2021

30 citations recorded by the source corpus.

Meta-learning promises few-shot learners that can adapt to new distributions by repurposing knowledge acquired from previous training. However, we believe meta-learning has not yet succeeded in NLP due to the lack of a well-defined task distribution, leading to attempts that treat datasets as tasks. Such an ad hoc task distribution causes problems of quantity and quality. Since there’s only a handful of datasets for any NLP problem, meta-learners tend to overfit their adaptation mechanism and, since NLP datasets are highly heterogeneous, many learning episodes have poor transfer between their support and query sets, which discourages the meta-learner from adapting. To alleviate these issues, we propose DReCA (Decomposing datasets into Reasoning Categories), a simple method for discovering and using latent reasoning categories in a dataset, to form additional high quality tasks. DReCA works by splitting examples into label groups, embedding them with a finetuned BERT model and then clustering each group into reasoning categories. Across four few-shot NLI problems, we demonstrate that using DReCA improves the accuracy of meta-learners by 1.5-4%

Subfields: Artificial Intelligence, Natural Language Processing, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiang Ren
2. Bill Yuchen Lin
3. Luke S. Zettlemoyer
4. Qinyuan Ye
5. Hannaneh Hajishirzi
6. Ruiqi Zhong
7. Zheng Zhang
8. Daniel Klein

In Natural Language Processing

1. Xiang Ren
2. Bill Yuchen Lin
3. Qinyuan Ye
4. Ruiqi Zhong
5. Zheng Zhang
6. Kristy Lee
7. Daniel Klein
8. Luke S. Zettlemoyer

In Meta & Continual Learning

1. Xiang Ren
2. Qinyuan Ye
3. Zheng Zhang
4. Daniel Klein
5. Bill Yuchen Lin

Citations in the corpus

• Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021, methodology): “Relation to Meta/Multitask-Learning Our method is closely related to, but different from meta-learning (Yin, 2020; Murty et al., 2021) and multi-task learning (Ye et al.”
• CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, methodology): “, 2020); (3) applying task augmentation strategies to prevent over-fitting (Murty et al., 2021); (4) learning to accumulate knowledge and avoid catastrophic forgetting in an continual learning setup (Jin et al.”