InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning

By Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh, Shikib Mehri, Maxine Eskénazi, Jeffrey P. Bigham. 2022

58 citations recorded by the source corpus.

Instruction tuning is an emergent paradigm in NLP wherein natural language instructions are leveraged with language models to induce zero-shot performance on unseen tasks. Dialogue is an especially interesting area in which to explore instruction tuning because dialogue systems perform multiple kinds of tasks related to language (e.g., natural language understanding and generation, domain-specific interaction), yet instruction tuning has not been systematically explored for dialogue-related tasks. We introduce InstructDial, an instruction tuning framework for dialogue, which consists of a repository of 48 diverse dialogue tasks in a unified text-to-text format created from 59 openly available dialogue datasets. We explore cross-task generalization ability on models tuned on InstructDial across diverse dialogue tasks. Our analysis reveals that InstructDial enables good zero-shot performance on unseen datasets and tasks such as dialogue evaluation and intent detection, and even better performance in a few-shot setting. To ensure that models adhere to instructions, we introduce novel meta-tasks. We establish benchmark zero-shot and few-shot performance of models trained using the proposed framework on multiple dialogue tasks.

Subfields: Artificial Intelligence, Natural Language Processing, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Prompting & Reasoning

Citations in the corpus

Self-Instruct: Aligning Language Models with Self-Generated Instructions (2022, background): “A series of works have found evidence that vanilla language models can be effective at following general language instructions if tuned with annotated “instructional” data – datasets containing language instructional commands and their desired outcome based on human judgement (Weller et al., 2020; M”
Language models show human-like content effects on reasoning (2022, background): “More broadly, language models can be trained or tuned to better follow instructions (Wei et al., 2021; Ouyang et al., 2022; Gupta et al., 2022).”
Style Over Substance: Evaluation Biases for Large Language Models (2023, unknown): “It is found that instruction-tuned LLMs can significantly enhance their performance following general language instructions (Weller et al., 2020; Mishra et al., 2022; Wang et al., 2022b; Wei et al., 2022; Sanh et al., 2022; Ouyang et al., 2022; Parmar et al., 2022; Scialom et al., 2022; Chung et al.”
HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models (2022, methodology): “Efficacy of the Prompting/Learning from Instruction techniques has been shown across diverse applications (Wang et al., 2022c) such as dialog (Gupta et al., 2022), NER (Wang et al., 2022a), program synthesis (Kuznia et al., 2022), style transfer (Reif et al., 2021), tabular question answering (Luo…”
Learning Performance-Improving Code Edits (2023, unknown): “Dataset imbalance.”
RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model (2023, background): “Since the introduction of GPT-3 [73], prompt learning has evolved from traditional discrete [69] and continuous prompt construction [59], [70] to large-scale model-oriented in-context learning [32], instruction-tuning [74], [75], [76], and chain-of-thought approaches [77], [78], [79].”