Multi-objective reinforcement learning using sets of pareto dominating policies

By Kristof Van Moffaert, Ann Nowé. Journal of machine learning research, 2014

412 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Renewable Energy, Sustainability and the Environment

View Paper: https://www.semanticscholar.org/paper/ec85eeb27f71f389416ebbf6e13c725ddef78bea

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Pieter Abbeel
3. Cosmin Paduraru
4. Chelsea Finn
5. Gabriel Dulac-Arnold
6. Ofir Nachum
7. Daniel Jaymin Mankowitz
8. Todd Hester

In Computational Theory and Mathematics

1. Sam Kwong
2. Gabriel Dulac-Arnold
3. Nicolas Manfred Otto Heess
4. Sergey Levine
5. Daniel Jaymin Mankowitz
6. J. Kolter
7. Leonard Hasenclever
8. Martin A. Riedmiller

In Renewable Energy, Sustainability and the Environment

1. J. Kolter
2. Fei Wang

Citations in the corpus

• A Distributional View on Multi-Objective Policy Optimization (2020, methodology): “Other approaches learn a set of policies simultaneously, by using a multi-objective variant of the Q-learning update rule (Barrett & Narayanan, 2008; Moffaert & Nowé, 2014; Reymond & Nowé, 2019; Yang et al., 2019) or by modifying gradientbased policy search (Parisi et al., 2014; Pirotta et al.,…”
• An empirical investigation of the challenges of real-world reinforcement learning (2020, background): “…with the multi-objective nature of the learning problems, either by predicting a value function for each objective [Van Seijen et al. 2017], or by finding a policy that optimizes each sub-problem [Li et al. 2019], or that fits each Pareto-dominating mixture of objectives [Moffaert and Now 2014].”
• Policy invariance under reward transformations for multi-objective reinforcement learning (2017, methodology): “Examples of such algorithms include Convex Hull Value Iteration [25], Pareto Q-Learning [26] and the work of Shelton [27].”
• Pareto Multi-Task Learning (2019, methodology): “It has been used in many machine learning applications such as reinforcement learning [15], Bayesian optimization [16–18] and neural architecture search [10, 19].”
• Reward Constrained Policy Optimization (2018, background): “This approach is a multi-objective problem (Mannor and Shimkin, 2004); in which for each set of penalty coefficients, there exists a different, optimal solution, also known as Pareto optimality (Van Moffaert and Now´e, 2014).”
• Batch Policy Learning under Constraints (2019, background): “Generally, research in MORL has largely focused on approximating the Pareto frontier that trades-off competing objectives (Van Moffaert & Nowé, 2014; Roijers et al., 2013).”
• Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space (2020, methodology): “…by the success of multi-objective optimization (MOO) problem [5, 25, 4, 20], we formulate the ensemble KD as a MOO problem with each objective corresponding to each teacher, which has been applied to recommender system [18], multi-task learning [26, 16], and multi-agent reinforcement learning [31].”
• Constrained Reinforcement Learning Has Zero Duality Gap (2019, background): “First, for each set of penalty coefficients, there exists a different, optimal solution, also known as Pareto optimality [6].”
• Task-based End-to-end Model Learning in Stochastic Optimization (2017, background): “Also highly related to our work are recent efforts in end-to-end policy learning [14], using value iteration effectively as an optimization procedure in similar networks [15], and multi-objective optimization [16, 17, 18, 19].”