PettingZoo: Gym for Multi-Agent Reinforcement Learning

By Justin K. Terry, Benjamin Black, Nathaniel Grammel, M. Jayakumar, Ananth Hari, Ryan Sullivan, L. Santos, R. Pérez, Caroline Horsch, Clemens Dieffendahl and 3 more. 2020

447 citations recorded by the source corpus.

This paper introduces the PettingZoo library and the accompanying Agent Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets of multi-agent environments with a universal, elegant Python API. PettingZoo was developed with the goal of accelerating research in Multi-Agent Reinforcement Learning ("MARL"), by making work more interchangeable, accessible and reproducible akin to what OpenAI's Gym library did for single-agent reinforcement learning. PettingZoo's API, while inheriting many features of Gym, is unique amongst MARL APIs in that it's based around the novel AEC games model. We argue, in part through case studies on major problems in popular MARL environments, that the popular game models are poor conceptual models of games commonly used in MARL and accordingly can promote confusing bugs that are hard to detect, and that the AEC games model addresses these problems.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/3a70562df004e08d91b125e6d15255e31e445efa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Gymnasium: A Standard Interface for Reinforcement Learning Environments (2025, background): “PettingZoo (Terry et al., 2021) is designed for multi-agent RL environments, offering a suite of environments where multiple agents can interact simultaneously.”
Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning (2020, methodology): “Similarly based on a preprint, all three methods were successfully experimentally together and separately in (Terry et al., 2020b).”
CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms (2022, background): “The following table reports the final episodic length instead of episodic return obtained by the agent in PettingZoo (Terry et al., 2021):”
Dealing with Non-Stationarity in MARL via Trust-Region Decomposition (2021, background): “The evaders move randomly, and the pursuers are controlled (Terry et al., 2020a).”