Reinforcement Learning with Factored States and Actions

By Brian Sallans, Geoffrey E. Hinton. 2004

188 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, methodology): “A closely related idea is presented in the work on energy-based reinforcement learning (Sallans and Hinton, 2004; Heess et al., 2013), where the free energy of an energy-based model (in that case, a restricted Boltzmann machine) is adjusted based on a reinforcement learning update rule, such that…”
Reinforcement Learning with Deep Energy-Based Policies (2017, methodology): “A number of prior works have also explored representing policies as energy-based models, with the Q-value obtained from an energy model such as a restricted Boltzmann machine (RBM) (Sallans & Hinton, 2004; Elfwing et al., 2010; Otsuka et al., 2010; Heess et al., 2012).”
PGQ: Combining policy gradient and Q-learning (2016, methodology): “In Sallans & Hinton (2004) the authors use a Boltzmann exploration policy over estimated Q-values which they update using TD-learning.”
Deep Reinforcement Learning with Double Q-Learning (2015, methodology): “This is important because ﬂexi-ble parametric function approximators are often employed in reinforcement learning (see, e.g., Tesauro 1995; Sallans and Hinton 2004; Riedmiller 2005; Mnih et al. 2015).”
Deep spatial autoencoders for visuomotor learning (2015, methodology): “Other approaches instead use function approximation to replace the value function in standard RL algorithms like TD-learning [16] and approximate Q-learning [17].”
Discrete Sequential Prediction of Continuous Actions for Deep RL (2017, background): “[36] Brian Sallans and Geoffrey E Hinton.”
Exploiting Structure and Agent-Centric Rewards to Promote Coordination in Large Multiagent Systems (2014, background): “Factored Markov Decision Processes (FMDPs) represent one approach to generalization, which exploits the underlying structure of the problem in multiagent systems [31].”
Learning Action Representations for Reinforcement Learning (2019, methodology): “A similar binary based approach was also used as an ensemble method to learning optimal policies for MDPs with large action sets [Sallans and Hinton, 2004].”
Free-energy-based reinforcement learning in a partially observable environment (2010, methodology): “We brieﬂy review the FERL framework for MDPs [3].”
Learning Visual Feature Spaces for Robotic Manipulation with Deep Spatial Autoencoders (2015, methodology): “Other approaches instead use function approximation to replace the value function in standard RL algorithms like TD-learning [16] and approximate Q-learning [17].”
Actor-Critic Reinforcement Learning with Energy-Based Policies (2012, methodology): “Free-energy Sarsa Sallans and Hinton (2004) use the RBM free energy as a non-linear function approximator.”
Conjugate Markov Decision Processes (2011, background): “Wire-fitting (Baird & Klopf, 1993) and Product of Experts (Sallans & Hinton, 2004) are not applicable because the actions are discrete and state continuous, respectively.”
VIREL: A Variational Inference Framework for Reinforcement Learning (2018, background): “[48] Sallans, B.”
Energy-Based Imitation Learning (2020, methodology): “Thanks to the appealing features, EBMs have been introduced into many RL literature, for instance, parameterized as value function [36], employed in the actor-critic framework [21], applied to MaxEnt RL [19] and modelbased RL regularization [3].”