Increasing generality in machine learning through procedural content generation

By Sebastian Risi, Julian Togelius. 2019

149 citations recorded by the source corpus.

Procedural content generation (PCG) refers to the practice of generating game content, such as levels, quests or characters, algorithmically. Motivated by the need to make games replayable, as well as to reduce authoring burden and enable particular aesthetics, many PCG methods have been devised. At the same time that researchers are adapting methods from machine learning (ML) to PCG problems, the ML community has become more interested in PCG-inspired methods. One reason for this development is that ML algorithms often only work for a particular version of a particular task with particular initial parameters. In response, researchers have begun exploring randomization of problem parameters to counteract such overfitting and to allow trained policies to more easily transfer from one environment to another, such as from a simulated robot to a robot in the real world. Here we review existing work on PCG, its overlap with current efforts in ML, and promising new research directions such as procedurally generated learning environments. Although originating in games, we believe PCG algorithms are critical to creating more general machine intelligence. This review covers the history of procedural content generation (PCG) approaches for video games, and how these approaches are now used to generate training data and environments for machine learning models. The authors then discuss how PCG may be crucial for training agents which generalise well.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In Meta & Continual Learning

Citations in the corpus

Choose Your Weapon: Survival Strategies for Depressed AI Academics (2023, background): “Procedural content generation for games was a very niche topic 15 years ago and we helped nuild a research community around it [24], [28]; recently it has become more important not only for the games industry, but also as a way to help generalize (deep) reinforcement learning [18], [23].”
Replay-Guided Adversarial Environment Design (2021, background): “DR itself is subsumed by procedural content generation [30], for which UED and DCD may be seen as providing a formal, decision-theoretic framework, enabling development of provably optimal algorithms.”
L2Explorer: A Lifelong Reinforcement Learning Assessment Environment (2022, background): “There are existing attempts to address this with procedurally generated open worlds (Risi and Togelius 2020), which generate tasks using randomly parameterized environments.”
Genie: Generative Interactive Environments (2024, unknown): “Environment generation Our work is also re-lated to Procedural Content Generation (PCG, e.g. Risi and Togelius, 2020a,b) where machine learning has proven highly effective for generating game levels (Summerville et al., 2018), recently via language models that directly write game code (Sudhakaran…”
Human-Timescale Adaptation in an Open-Ended Task Space (2023, methodology): “PCG has been studied for many years in the games community (Risi and Togelius, 2020; Togelius and Schmidhuber, 2008) and more recently has been used to create testbeds for RL agents (Cobbe et al.”
Prioritized Level Replay (2020, background): “…process that, given a level identiﬁer (e.g. an index or a random seed), generates a level , deﬁned as an environment instance exhibiting a unique conﬁguration of its underlying factors of variation, such as layout, asset appearances, or speciﬁc environment dynamics (Risi & Togelius, 2020).”