Simple random search of static linear policies is competitive for reinforcement learning

By Horia Mania, Aurelia Guy, B. Recht. 2018

309 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “A recent study in [95] demonstrates that random search over the parameters for a policy network can perform as well as LQR.”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “…experiments that, when collecting large amount of on-policy data is not an issue, direct parameter search methods such as augmented random search (Mania et al. 2018) can often be substantially easier to deploy than more classic RL methods, likely to their ability to avoid local optima by…”
A Deep Reinforcement Learning Perspective on Internet Congestion Control (2019, methodology): “We also tried simple random search and hill-climbing for linear models, whic (Mania et al., 2018) recently showed performs well on continuous Mujoco tasks, but these were not competitive in our context.”
Fine-Tuning Language Models with Just Forward Passes (2023, methodology): “There are also ZO methods that optimize without estimating the gradient [38, 68, 44].”
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents (2021, background): “Some of these tasks have been shown to be easily solvable by random search algorithms (Mania et al., 2018) and thus should not be considered as sufficiently difficult for comparing algorithms.”
Neuroevolution of self-interpretable agents (2020, unknown): “Neuroevolution-based methods for tackling challenging RL tasks have recently gained popularity due to their simplicity and competitiveness to Deep RL methods, even on vision-based RL benchmark tasks [34, 36, 62, 81, 94].”
Learning to Score Behaviors for Guided Policy Optimization (2019, background): “This approach has gained in popularity recently [30, 20, 6].”
Square Attack: a query-efficient black-box adversarial attack via random search (2019, unknown): “Random search has been successfully applied to reinforcement learning [32] where it came out to be competitive to gradient-based methods.”
Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation (2021, background): “This sort of scaling is expected: policy gradient algorithms are well known to require significantly more environment frames to estimate a gradient than simply calculating the gradient directly [9].”
Simplifying Deep Reinforcement Learning via Self-Supervision (2021, methodology): “While these meth-ods are shown to have a competitive performance on linear model (Mania, Guy, and Recht 2018), they are sample intensive when using deep models (Salimans et al. 2017).”
Neural Lyapunov Control (2020, background): “There is strong evidence that linear control functions are all we need for solving highly nonlinear control problems through reinforcement learning as well [21], suggesting convergence of different learning approaches.”