Memory Grounded Conversational Reasoning

By Seungwhan Moon, Pararth Shah, Rajen Subba, Anuj Kumar. 2019

5 citations recorded by the source corpus.

We demonstrate a conversational system which engages the user through a multi-modal, multi-turn dialog over the user’s memories. The system can perform QA over memories by responding to user queries to recall specific attributes and associated media (e.g. photos) of past episodic memories. The system can also make proactive suggestions to surface related events or facts from past memories to make conversations more engaging and natural. To implement such a system, we collect a new corpus of memory grounded conversations, which comprises human-to-human role-playing dialogs given synthetic memory graphs with simulated attributes. Our proof-of-concept system operates on these synthetic memory graphs, however it can be trained and applied to real-world user memory data (e.g. photo albums, etc.) We present the architecture of the proposed conversational system, and example queries that the system supports.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In LLM Agents & Tool Use

Citations in the corpus

Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “The current approaches to using long-term knowledge are either graph representations (Moon et al., 2019a) or unstructured text retrieval (Chen et al., 2017; Dodge et al., 2016; Dinan et al., 2019b), which is then prepended onto the dialogue history and attended over, an approach advocated by the…”
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents (2019, background): “For example, longer conversations involving memory (Moon et al., 2019), or mixing open-domain conversation with task oriented goals.”