Distributed Value Functions

By Jeff Schneider, Weng-Keen Wong, Andrew W. Moore, Martin A. Riedmiller. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, background): “1.1 Other Related Work Schneider et al. (1999) consider the optimization of the sum of individual reward functions, by optimizing local compositions of individual value functions learnt from them.”
A Survey of Multi-Objective Sequential Decision-Making (2013, result): “This work is similar to that of Schneider, Wong, Moore, and Riedmiller (1999), which addresses the use of multiple agents in a distributed network such as a power distribution grid, where the aim is to maximize a global reward formed from a combination of each agent’s local reward.”
Coordinated Reinforcement Learning (2002, background): “One natural approach to reducing the complexity of this problem is to restrict the amount of information that is available to each agent and hope to maximize global welfare by solving local optimization problems for each agent [14].”
Collaborative Multiagent Reinforcement Learning by Payoff Propagation (2006, methodology): “5 Distributed Value Functions Another method to decompose a large action space is the distributed value functions (DVF) approach (Schneider et al., 1999).”
Coordinated Multi-Robot Exploration Under Communication Constraints Using Decentralized Markov Decision Processes (2012, background): “Distributed value functions (DVF) have been introduced by (Schneider et al. 1999) as a way to distribute reinforcement learning knowledge through different agents in the case of distributed systems.”
An Introduction to Multi-Agent Systems (2010, unknown): “In [64], distributed value function based on RL has been described.”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, background): “transportation schedule [22, 53, 159, 253], traffic signal control [50, 59], energy management [180], wireless security [172], satellite docking [76], edge computing [299], chemical processes [241], video games [28, 153, 192, 211, 262, 300], board games of Go, shogi, chess and arcade game PAC-MAN [1”
Multiagent learning is not the answer. It is the question (2007, unknown): “A partial list of other examples includes collaborative multi-robot localization [2], distributed network routing [4], distributed factory optimization [ 9 ], in-city driving [8], tracking teams of enemy combatants [13], and bidding in auctions [12].”
Reinforcement learning for stochastic cooperative multi-agent-systems (2004, background): “…Martin Lauer Martin Riedmiller University of Osnabr ¨uck 49069 Osnabr ¨uck, Germany Abstract We present a distributed variant of Q-learning that allows to learn the optimal cost-to-go function in stochastic cooperative multi-agent domains without communication between the agents.”
Hierarchical multi-agent reinforcement learning (2001, methodology): “This idea has been addressed using value function based RL [ 34 ] as well as policy gradient based RL [29].”
Distributed Policy Evaluation Under Multiple Behavior Strategies (2013, unknown): “Other related—but more heuristic—approaches include [28], [29].”
Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges (2017, background): “Various kinds of information can be exchanged including the Q-value of the current action [159] and the maximum Q-value of the current state (also called value function) [160].”
Learning to Cooperate via Policy Search (2000, background): “[15] investigate a case of distributed reinforcement learning, in which agents have complete and reliable state observation, but only receive a local reinforcement signal.”
Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning (2020, background): “These include optimising over local compositions of individual Q-value functions learnt from individual reward functions (Schneider et al., 1999), as well as summing individualQ-functions with individual rewards before greedy joint action selection (Russell & Zimdars, 2003).”
Value-Decomposition Networks For Cooperative Multi-Agent Learning Based On Team Reward (2018, background): “[21] and Russell and Zimdars [20] optimize the sum of individual rewards by learning value functions from those individual rewards.”
Automated Design of Adaptive Controllers for Modular Robots using Reinforcement Learning (2008, methodology): “More recently there has been much theoretical and algorithmic work in multi-agent reinforcement learning, notably in cooperative reinforcement learning through distributed value functions ( Schneider et al. 1999 ), coordination graphs (Guestrin et al. 2002, Kok & Vlassis 2006), and individual reward”
Multiagent Planning with Factored MDPs (2001, methodology): “Alternative approaches to this problem have used local optimization for the different agents, either via reward/value sharing [11, 13] or direct policy search [10].”