GODEL: Large-Scale Pre-Training for Goal-Directed Dialog

By Baolin Peng, Michel Galley, Pengcheng He, Chris Brockett, Lars Lidén, Elnaz Nouri, Zhou Yu, Bill Dolan, Jianfeng Gao. 2022

80 citations recorded by the source corpus.

We introduce GODEL (Grounded Open Dialogue Language Model), a large pre-trained language model for dialog. In contrast with earlier models such as DialoGPT, GODEL leverages a new phase of grounded pre-training designed to better support adapting GODEL to a wide range of downstream dialog tasks that require information external to the current conversation (e.g., a database or document) to produce good responses. Experiments against an array of benchmarks that encompass task-oriented dialog, conversational QA, and grounded open-domain dialog show that GODEL outperforms state-of-the-art pre-trained dialog models in few-shot fine-tuning setups, in terms of both human and automatic evaluation. A novel feature of our evaluation methodology is the introduction of a notion of utility that assesses the usefulness of responses (extrinsic evaluation) in addition to their communicative features (intrinsic evaluation). We show that extrinsic evaluation offers improved inter-annotator agreement and correlation with automated metrics. Code and data processing scripts are publicly available.

Subfields: Artificial Intelligence, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Dialogue & Conversational AI

Citations in the corpus

Using In-Context Learning to Improve Dialogue Safety (2023, background): “Large neural-based language models are becoming increasingly proficient dialogue agents (Roller et al., 2021; Peng et al., 2022; Thoppilan et al., 2022; Touvron et al., 2023).”
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback (2023, methodology): “We further include BLEURT (Sellam et al., 2020), BERTScore (Zhang et al., 2019), chrF (Popovi´c, 2015), which have been shown to be among the best-performing text generation metrics on dialog(Yeh et al., 2021; Peng et al., 2022).”
BloombergGPT: A Large Language Model for Finance (2023, unknown): “…et al., 2022; Tafjord and Clark, 2021; Smith et al., 2022; Scao et al., 2022; Taylor et al., 2022; Lin et al., 2022; Soltan et al., 2022; Thoppilan et al., 2022; Bao et al., 2022; Sanh et al., 2022; Roller et al., 2021; Glaese et al., 2022; Wang et al., 2021; Peng et al., 2022, among many others).”