Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations

By Yordan Yordanov, Vid Kocijan, Thomas Lukasiewicz, Oana-Maria Camburu. 2021

22 citations recorded by the source corpus.

Training a model to provide natural language explanations (NLEs) for its predictions usually requires the acquisition of task-specific NLEs, which is time- and resource-consuming. A potential solution is the few-shot out-of-domain transfer of NLEs from a parent task with many NLEs to a child task. In this work, we examine the setup in which the child task has few NLEs but abundant labels. We establish four few-shot transfer learning methods that cover the possible fine-tuning combinations of the labels and NLEs for the parent and child tasks. We transfer explainability from a large natural language inference dataset (e-SNLI) separately to two child tasks: (1) hard cases of pronoun resolution, where we introduce the small-e-WinoGrande dataset of NLEs on top of the WinoGrande dataset, and (2)~commonsense validation (ComVE). Our results demonstrate that the parent task helps with NLE generation and we establish the best methods for this setup.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fan Yang
2. Ninghao Liu
3. Mengnan Du
4. Huiqi Deng
5. Shuaiqiang Wang
6. Haiyan Zhao
7. Hengyi Cai
8. Dawei Yin

In Computer Vision and Pattern Recognition

1. Fan Yang
2. Mengnan Du
3. Huiqi Deng
4. Shuaiqiang Wang
5. Haiyan Zhao
6. Hengyi Cai
7. Ninghao Liu
8. Dawei Yin

In Natural Language Processing

1. Huiqi Deng
2. Shuaiqiang Wang
3. Haiyan Zhao
4. Hengyi Cai
5. Mengnan Du
6. Ninghao Liu
7. Fan Yang
8. Dawei Yin

Citations in the corpus

• Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “That line of work typically focuses on natural language inference (Camburu et al., 2018; Yordanov et al., 2021; Bostrom et al., 2021), and produces explanations either simultaneously to or after the final prediction (Narang et al., 2020; Majumder et al., 2021; Wiegreffe et al., 2021, 2022).”
• Explainability for Large Language Models: A Survey (2023, background): “It first represents the concept with a set of examples and then learns a linear classifier as “ concept activation vector ”( CAV ) to detect interested concept.”