Particle swarm optimization for generating interpretable fuzzy reinforcement learning policies

By Daniel Hein, Alexander Hentschel, Thomas A. Runkler, Steffen Udluft. 2016

89 citations recorded by the source corpus.

Abstract Fuzzy controllers are efficient and interpretable system controllers for continuous state and action spaces. To date, such controllers have been constructed manually or trained automatically either using expert-generated problem-specific cost functions or incorporating detailed knowledge about the optimal control strategy. Both requirements for automatic training processes are not found in most real-world reinforcement learning (RL) problems. In such applications, online learning is often prohibited for safety reasons because it requires exploration of the problem’s dynamics during policy training. We introduce a fuzzy particle swarm reinforcement learning (FPSRL) approach that can construct fuzzy RL policies solely by training parameters on world models that simulate real system dynamics. These world models are created by employing an autonomous machine learning technique that uses previously generated transition samples of a real system. To the best of our knowledge, this approach is the first to relate self-organizing fuzzy controllers to model-based batch RL. FPSRL is intended to solve problems in domains where online learning is prohibited, system dynamics are relatively easy to model from previously generated default policy transition samples, and it is expected that a relatively easily interpretable control policy exists. The efficiency of the proposed approach with problems from such domains is demonstrated using three standard RL benchmarks, i.e., mountain car, cart-pole balancing, and cart-pole swing-up. Our experimental results demonstrate high-performing, interpretable fuzzy policies.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Evaluating Reinforcement Learning Algorithms in Observational Health Settings (2018, background): “More recently some works have also speciﬁ-cally attempted to address the problem of interpretability in RL algorithms [Maes et al., 2012, Hein et al., 2017].”
Toward Interpretable-AI Policies Using Evolutionary Nonlinear Decision Trees for Discrete-Action Systems (2020, methodology): “Particle swarm optimization [20] is used to generate the interpretable fuzzy rule set in [21] and is demonstrated on classic control problems involving continuous actions.”
Interpretable Policies for Reinforcement Learning by Genetic Programming (2017, methodology): “Recent research results show that using fuzzy rules in batch RL settings can be considered an adequate solution to this task (Hein, Hentschel, Runkler, and Udluft, 2017b).”
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches (2018, background): “Recently Fuzzy RL policies have been proposed that is suitable for continuous state and action space [251].”