Learning functions across many orders of magnitudes

17 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Red Teaming Language Models with Language Models (2022, methodology): “We use PopArt (van Hasselt et al., 2016) with a step size of 10−4 to account for variation in the value function scale during learning.”
Learning values across many orders of magnitude (2016, methodology): “In the Atari domain several algorithmic variants and improvements for DQN have been proposed [van Hasselt et al., 2016, Bellemare et al., 2016, Schaul et al., 2016, Wang et al., 2016], as well as alternative solutions [Liang et al., 2016, Mnih et al., 2016].”
Neural Episodic Control (2017, methodology): “A number of extensions have been proposed that improve DQN. Double DQN (Van Hasselt et al., 2016) reduces bias on the target calculation.”
V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control (2019, methodology): “As we are interested in the multi-task setting where a single agent must learn a large number of tasks with differing reward scales, we used PopArt (van Hasselt et al., 2016; Hessel et al., 2018) for the value function, even when training on a single task.”
Evaluating Rewards for Question Generation Models (2019, methodology): “All rewards were normalised with a simple form of PopArt (Hasselt et al., 2016), with the running mean µ R and standard deviation σ R updated on-line during training.”