Multitasking Inhibits Semantic Drift

By Athul Paul Jacob, Michael Lewis, Jacob Andreas. 2021

13 citations recorded by the source corpus.

When intelligent agents communicate to accomplish shared goals, how do these goals shape the agents’ language? We study the dynamics of learning in latent language policies (LLPs), in which instructor agents generate natural-language subgoal descriptions and executor agents map these descriptions to low-level actions. LLPs can solve challenging long-horizon reinforcement learning problems and provide a rich model for studying task-oriented language use. But previous work has found that LLP training is prone to semantic drift (use of messages in ways inconsistent with their original natural language meanings). Here, we demonstrate theoretically and empirically that multitask training is an effective counter to this problem: we prove that multitask training eliminates semantic drift in a well-studied family of signaling games, and show that multitask training of neural LLPs in a complex strategy game reduces drift and while improving sample efficiency.

Subfields: Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Science & Psychology

1. Antonio Torralba
2. Yilun Du
3. Adam Lerer
4. Ekin Akyürek
5. Jacob Andreas
6. Daniel Fried

Citations in the corpus

• Pre-Trained Language Models for Interactive Decision-Making (2022, background): “Semi-Supervised Skill Learning with Latent Language (SL) 3 [38] shows that LMs can serve as an effective backbone for hierarchical policies that express plans as natural language strings [2, 4].”
• Skill Induction and Planning with Latent Language (2021, background): “, 2018), restricted language (Hu et al., 2019; Jacob et al., 2021) or synthetic training data (Shu et al.”
• Human-level play in the game of Diplomacy by combining language models with strategic reasoning (2022, result): “This effect can be clearly seen in settings that involve language, in which prior work found that self-play produced uninterpretable language despite achieving high task success for the agents ( 14 , 15 ).”
• One-Shot Learning from a Demonstration with Hierarchical Latent Language (2022, unknown): “More recent work has applied LLPs to real-time strategy games (Hu et al., 2019; Jacob et al., 2021), while Chen et al. (2021) show that LLPs trained to generate and follow crowdsourced instructions can perform few-or zero-shot simple crafting tasks in a small grid world.”