Learning Through Dialogue Interactions

By Jiwei Li, Alexander H. Miller, Sumit Chopra, Marc'Aurelio Ranzato, Jason Aaron Edward Weston. 2016

55 citations recorded by the source corpus.

A good dialogue agent should have the ability to interact with users by both responding to questions and by asking questions, and importantly to learn from both types of interaction. In this work, we explore this direction by designing a simulator and a set of synthetic tasks in the movie domain that allow such interactions between a learner and a teacher. We investigate how a learner can benefit from asking questions in both offline and online reinforcement learning settings, and demonstrate that the learner improves when asking questions. Finally, real experiments with Mechanical Turk validate the approach. Our work represents a first step in developing such end-to-end learned interactive dialogue agents.

Subfields: Artificial Intelligence, Natural Language Processing, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In LLM Agents & Tool Use

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, background): “Dialog based learning (Weston, 2016; Li et al., 2016; Hancock et al., 2019) develops techniques to learn from interactive natural language feedback in dialog.”
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “In order to solve this two strategies have been tried: trying to learn to model the personality as part of the learned weights given the speaker id from the data (Li et al., 2016), or providing training data with explicit personality information.”
ExpBERT: Representation Engineering with Natural Language Explanations (2020, background): “While the above works consider learning from static language supervision, Li et al. (2016) and Weston (2016) learn from language supervision in an interactive setting.”
Training Classifiers with Natural Language Explanations (2018, background): “Weston (2016); Li et al. (2016) learn from natural language feedback in a dialogue.”
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, background): “Li et al. (2016b) studies models that learn how to sometimes ask appropriate questions in order to learn from the answers, while Li et al. (2016a) learns from general textual feedback/comments from the user, particularly in the case where the bot has produced a low quality response.”
Neural Approaches to Conversational AI (2018, positive): “erstanding and Dialogue State Tracking NLU and DST are two closely related components essential to a dialogue system. They can have a signiﬁcant impact on the overall system’s performance (see, e.g., Li et al. (2017e)). This section reviews some of the classic and state-of-the-art approaches. 4.3.1 ”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, background): “[41] deﬁned three situations where the bot has problems in answering a question.”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “One is agenda-based simulators (Schatzmann and Young, 2009; Li et al., 2016e; Ultes et al., 2017), which only feed dialogue systems with the pre-defined user goal as a user message, without surface realization.”
Learning to Caption Images Through a Lifetime by Asking Questions (2018, background): “In [11], the agent learns whether to ask questions to the teacher to efficiently solve dialogue tasks.”
Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback (2020, methodology): “Our work is linked to multiple existing research lines including semantic parsing, learning through interaction (Li et al., 2017a; Hancock et al., 2019; Li et al., 2017b, inter alia) and learning from natural language supervision (Srivastava et al., 2017; CoReyes et al., 2019; Srivastava et al.,…”
LIREx: Augmenting Language Inference with Relevant Explanation (2020, background): “Studies in this direction include (Li et al. 2016; Srivastava, Labutov, and Mitchell 2017; Wang et al. 2017; Hancock et al. 2018; Qin et al. 2020; Lee et al. 2020), in which the authors used semantic parsers to convert unstructured NLEs into structured feature-like logical forms.”
Why Build an Assistant in Minecraft? (2019, methodology): “This has been studied in (ungrounded) conversational systems [77, 90, 68, 49] and when grounding with images via the CLEVR QA task [62].”
Lifelong and Interactive Learning of Factual Knowledge in Dialogues (2019, background): “on rules and information retrieval techniques (Banchs and Li, 2012; Ameixa et al., 2014; Lowe et al., 2015; Serban et al., 2015). Recent deep learning models (Vinyals and Le, 2015; Xing et al., 2017; Li et al., 2017c) learn from large corpora. However, since they do not use explicit knowledge bases ”
Lifelong Knowledge Learning in Rule-based Dialogue Systems (2020, background): “Recently, deep learning models are the mainstream in research (Vinyals and Le, 2015; Xing et al., 2017; Li et al., 2017, Shu et al, 2019).”
Continuous and Interactive Factual Knowledge Learning in Veriﬁcation Dialogues (2020, background): “Among dialogue systems [47, 25, 26, 10, 52, 57], the work in [26] allows the learner to ask questions.”
Towards Conversational Search and Recommendation: System Ask, User Respond (2018, background): “With the ability of logical inference on textual inputs, Memory Network (MemNN) [13, 32, 38] has been one of the state-of-theart approaches to dialog systems [4, 10, 19], as well as several other closely related tasks, including question answering [5, 17, 19, 40], language learning [39], and machine”
Towards a Continuous Knowledge Learning Engine for Chatbots (2018, background): “The work in (Li et al., 2016b) allows a learner to ask questions in dialogue.”