Autoencoder-augmented neuroevolution for visual doom playing
By Samuel Alvernaz, Julian Togelius. 2017
63 citations recorded by the source corpus.
Neuroevolution has proven effective at many re-inforcement learning tasks, including tasks with incomplete information and delayed rewards, but does not seem to scale well to high-dimensional controller representations, which are needed for tasks where the input is raw pixel data. We propose a novel method where we train an autoencoder to create a comparatively low-dimensional representation of the environment observation, and then use CMA-ES to train neural network controllers acting on this input data. As the behavior of the agent changes the nature of the input data, the autoencoder training progresses throughout evolution. We test this method in the VizDoom environment built on the classic FPS Doom, where it performs well on a health-pack gathering task.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- David Filliat
- Timothée Lesort
- Natalia Díaz Rodríguez
- David R Ha
- Sergey Levine
- J. Urgen Schmidhuber
- Alexandre Heuillet
- Fabien Couthouis
Citations in the corpus
- Recurrent World Models Facilitate Policy Evolution (2018, neutral): “Evolution-based algorithms have even been able to solve difficult RL tasks from high dimensional pixel inputs [1, 31, 45, 63].”
- World Models (2018, positive): “More recent works (Alvernaz & Togelius, 2017) combine VAE and ES, which is similar to our approach.”
- State Representation Learning for Control: An Overview (2018, methodology): “ES optimization methods have shown to be efficient for deep reinforcement learning [Conti et al., 2017] but they could then take a clear advantage of a lower dimension input to explore faster the parameter space than using raw data [Alvernaz and Togelius, 2017].”
- Explainability in Deep Reinforcement Learning (2020, methodology): “There are several approaches employed for SRL: reconstructing the observations using autoencoders [6, 32], training a forward model to predict next state [49, 80], teach to an inverse model how to predict actions from previous state(s) [95, 80] or using prior knowledge to constrain the state space [”
- Synthetic Data for Deep Learning (2019, background): “[314] and later used in many important works [15,343,480,648].”
- Automated Curriculum Learning by Rewarding Temporally Rare Events (2018, methodology): “A very different approach by Alvernaz and Togelius applies neuroevolution on top of a pre-trained auto-encoder [1].”