A benchmark environment motivated by industrial control problems

By Daniel Hein, Stefan Depeweg, Michel Tokic, Steffen Udluft, Alexander Hentschel, Thomas A. Runkler, Volkmar Sterzing. 2017

68 citations recorded by the source corpus.

In the research area of reinforcement learning (RL), frequently novel and promising methods are developed and introduced to the RL community. However, although many researchers are keen to apply their methods on real-world problems, implementing such methods in real industry environments often is a frustrating and tedious process. Generally, academic research groups have only limited access to real industrial data and applications. For this reason, new methods are usually developed, evaluated and compared by using artificial software benchmarks. On one hand, these benchmarks are designed to provide interpretable RL training scenarios and detailed insight into the learning process of the method on hand. On the other hand, they usually do not share much similarity with industrial real-world applications. For this reason we used our industry experience to design a benchmark which bridges the gap between freely available, documented, and motivated artificial benchmarks and properties of real industrial problems. The resulting industrial benchmark (IB) has been made publicly available to the RL community by publishing its Java and Python code, including an OpenAI Gym wrapper, on Github. In this paper we motivate and describe in detail the IB's dynamics and identify prototypic experimental settings that capture common situations in real-world industry control problems.

Subfields: Artificial Intelligence, Industrial and Manufacturing Engineering, Evaluation & Benchmarks, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Industrial and Manufacturing Engineering

In Evaluation & Benchmarks

In Robotics & Control

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, background): “These methods have demonstrated promising results on challenging control tasks [32], where the states well deﬁned, and the observation is relatively low dimensional.”
World Models (2018, background): “These methods have demonstrated promising results on challenging control tasks (Hein et al., 2017), where the states are known and well deﬁned, and the observation is relatively low dimensional.”
Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning (2017, background): “The first one is given by the industrial benchmark (IB), a publicly available simulator with properties inspired by real industrial systems (Hein et al., 2017).”
Interpretable Policies for Reinforcement Learning by Genetic Programming (2017, background): “For (ii), the oﬀline RL problem, model-based algorithms are usually more stable and yield better performing policies (Hein et al., 2017b).”
Decomposition of Uncertainty for Active Learning and Reliable Reinforcement Learning in Stochastic Systems (2017, methodology): “We show now the effectiveness of our proposed method the industrial benchmark, a stochastic system with properties inspired by real industrial systems [Hein et al., 2017].”
NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning (2021, methodology): “The industrial benchmark (IB) [32] is an RL benchmark environment motivated to simulate the characteristics presented in various industrial control tasks, such as wind or gas turbines, chemical reactors, etc.”