The Hanabi Challenge: A New Frontier for AI Research
By Nolan Bard, Jakob N. Foerster, A. P. Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes and 5 more. 2019
429 citations recorded by the source corpus.
Abstract From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.
Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning, Game & Decision Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Sergey Levine
- Mohit Bansal
- Jianfeng Gao
- Ruslan Salakhutdinov
- Pieter Abbeel
- Quoc V. Le
- Percy Liang
In Management Science and Operations Research
- Shimon Whiteson
- Jakob N. Foerster
- Noam Brown
- Adam Lerer
- Christian Schröder de Witt
- Tabish Rashid
- Neil Burch
- Yi Wu
In Reinforcement Learning
- Shimon Whiteson
- Jakob N. Foerster
- Adam Lerer
- Hengyuan Hu
- Christian Schröder de Witt
- Sergey Levine
- Tabish Rashid
- Chongjie Zhang
In Game & Decision Theory
- Noam Brown
- Jakob N. Foerster
- Adam Lerer
- Neil Burch
- Hengyuan Hu
- Michael H. Bowling
- Shimon Whiteson
- Pablo Hernandez-Leal
Citations in the corpus
- OpenSpiel: A Framework for Reinforcement Learning in Games (2019, neutral): “Game Reference(s) Backgammon Wikipedia Breakthrough Wikipedia Bridge bidding Wikipedia Catch [51] and [59, Appendix A] Coin Game [64] Connect Four Wikipedia Cooperative Box-Pushing [70] Chess Wikipedia First-price Sealed-bid Auction Wikipedia Go Wikipedia Goofspiel Wikipedia Hanabi (via HLE) Wikiped”
- The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games (2021, methodology): “9 Hanabi is a turn-based card game, introduced as a MARL challenge in (Bard et al., 2020) , where each agent observes other players’ cards except their own cards.”
- Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, neutral): “…et al., 2017; Yang et al., 2018; Zheng et al., 2017), there are currently no challenging standard testbeds for centralised training with decentralised execution with the exception of the recently introduced Hanabi Challenge (Bard et al., 2020) which focusses on a setting with less than 5 agents.”
- Continuous Coordination As a Realistic Scenario for Lifelong Learning (2021, methodology): “Hanabi (Bard et al., 2020) is a partially-observable, fully cooperative multiagent game that consists of 2-5 players.”
- Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings (2021, methodology): “While early hobbyist agents codified human conventions with some search (O’Dwyer, 2019; Wu, 2018), more recent work has focused on Han-abi as a challenge problem for learning in cooperative partially-observed games (Bard et al., 2020).”
- Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning (2019, methodology): “For a more comprehensive review on previous results on Hanabi we refer the reader to Bard et al. (2019).”
- Google Research Football: A Novel Reinforcement Learning Environment (2019, background): “For instance, the DeepMind Control Suite (Tassa et al., 2018) focuses on continuous control, the AI Safety Gridworlds (Leike et al., 2017) on learning safely, whereas the Hanabi Learning Environment (Bard et al., 2019) proposes a multi-agent setup. As a consequence, each of these environments are be”
- QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, methodology): “, 2018; Zheng et al., 2017), there are currently no challenging standard testbeds for centralised training with decentralised execution with the exception of the recently introduced Hanabi Challenge (Bard et al., 2020) which focusses on a setting with less than 5 agents. SMAC lls this gap. It is bui”
- Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning (2018, background): “BAD thus establishes a current state-of-the-art on the Hanabi Learning Environment (Bard et al., 2019) for the two player selfplay setting.”
- A survey and critique of multiagent deep reinforcement learning (2018, background): “nformation about them. This makes an interesting challenge for learning algorithms in particular in the context of self-play learning and ad-hoc teams [306, 307, 308]. The Hanabi Learning Environment [309] was recently released24 and it is accompanied with a baseline (deep RL) agent [310]. Arena [31”
- "Other-Play" for Zero-Shot Coordination (2020, methodology): “…zero-shot coordination problem, 2) we show that OP is the highest payoff meta-equilibrium for the zero-shot coordination problem, 3) we show how to implement OP using deep reinforcement learning (deep RL) based methods, and 4) we evaluate OP in the cooperative card game Hanabi (Bard et al., 2020).”
- Improving Policies via Search in Cooperative Partially Observable Games (2019, methodology): “…than current learning algorithms” because while RL algorithms perform exploration to find a good joint policy (convention), humans instead typically start with a convention and then individually search for the best action assuming that their partners will play the convention (Bard et al. 2019).”
- Social Diversity and Social Preferences in Mixed-Motive Reinforcement Learning (2020, background): “Examples of the former include games such as Hanabi [6] and Go [49].”
- Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration (2020, background): “There has been a large body of platforms for various multi-agent tasks (Jaderberg et al., 2019; Samvelyan et al., 2019; OpenAI, 2018; Lowe et al., 2017; Resnick et al., 2018; Shu & Tian, 2018; Carroll et al., 2019; Suarez et al., 2019; Baker et al., 2019; Bard et al., 2020).”
- Neural Replicator Dynamics (2019, neutral): “n of MARL agents [51, 54, 65– 67]. Connections have been made between MARL and EGT [6, 68], and population-based training regimes have been critical in scaling MARL to complex multiagent environments [4, 30, 70]. Central to EGT are the standard replicator dynamics (RD): a dynamical system 1Thisalgor”
- Re-determinizing Information Set Monte Carlo Tree Search in Hanabi (2019, background): “Hanabi [2] is a co-operative game for 2-5 players that has attracted some attention in games research due to the role of hidden information, a restricted communication channel and need to model one’s fellow players, most recently [3].”
- Revisiting the Evaluation of Theory of Mind through Question Answering (2019, background): “We believe this is a promising approach for advancing research on theory of mind, as it decouples its evaluation from problems such as multi-agent systems, game theory, and meta-learning, which are all components of other evaluation methods (Ra-binowitz et al., 2018; Bard et al., 2019).”