Batch reinforcement learning on the industrial benchmark: First experiences

By Daniel Hein, Steffen Udluft, Michel Tokic, Alexander Hentschel, Thomas A. Runkler, Volkmar Sterzing. 2017

12 citations recorded by the source corpus.

The Particle Swarm Optimization Policy (PSO-P) has been recently introduced and proven to produce remarkable results on interacting with academic reinforcement learning benchmarks in an off-policy, batch-based setting. To further investigate the properties and feasibility on real-world applications, this paper investigates PSO-P on the so-called Industrial Benchmark (IB), a novel reinforcement learning (RL) benchmark that aims at being realistic by including a variety of aspects found in industrial applications, such as continuous state and action spaces, a high dimensional, partially observable state space, delayed effects, and complex stochasticity. The experimental results of PSO-P on IB are compared to results of closed-form control policies derived from the model-based Recurrent Control Neural Network (RCNN) and the model-free Neural Fitted Q-Iteration (NFQ). Experiments show that PSO-P is not only of interest for academic benchmarks, but also for real-world industrial applications, since it also yielded the best performing policy in our IB setting. Compared to other well established RL techniques, PSO-P produced outstanding results in performance and robustness, requiring only a relatively low amount of effort in finding adequate parameters or making complex design decisions.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

D4RL: Datasets for Deep Data-Driven Reinforcement Learning (2020, methodology): “This evaluation protocol has been used in domains such as continuous control for robotics [Fujimoto et al., 2018, Kumar et al., 2019, Wu et al., 2019], navigation [Laroche et al., 2019], industrial control [Hein et al., 2017], and Atari video games [Agarwal et al., 2019b].”
A benchmark environment motivated by industrial control problems (2017, methodology): “Previously, this benchmark has already been used to demonstrate the performance of a particle swarm based RL policy approach [10].”
Datasets for Data-Driven Reinforcement Learning (2020, methodology): “, 2019], industrial control [Hein et al., 2017], and Atari video games [Agarwal et al.”
Interpretable Policies for Reinforcement Learning by Genetic Programming (2017, background): “For (ii), the oﬀline RL problem, model-based algorithms are usually more stable and yield better performing policies (Hein et al., 2017b).”