REINFORCEMENT DRIVEN INFORMATION ACQUISITION IN NON-DETERMINISTIC ENVIRONMENTS

By In Proc. Icann, Jan Storck, Sepp Hochreiter. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Reward learning from human preferences and demonstrations in Atari (2018, background): “…information between actions and end states (Gregor et al., 2016), state prediction error or surprise (Pathak et al., 2017), state visit counts (Storck et al., 1995; Bellemare et al., 2016), distinguishability to a decoder (Eysenbach et al., 2018), or empowerment (Salge et al., 2014), which is…”
Recurrent World Models Facilitate Policy Evolution (2018, background): “An exciting research direction is to look at ways to incorporate artiﬁcial curiosity and intrinsic motivation [61, 64, 77, 80, 81] and information seeking [23, 86] abilities in an agent to encourage novel exploration [47].”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “On the other hand, many data modalities are not observed as i.i.d. samples from a distribution (Dayan, 1993; Storck et al., 1995; Hochreiter & Schmidhuber, 1999; Bengio et al., 2013; Peters et al., 2017; Thomas et al., 2017; Schölkopf, 2019).”
World Models (2018, background): “…at ways to incorporate artiﬁcial curiosity and intrinsic motivation (Schmidhu-ber, 2010; 2006; 1991b; Pathak et al., 2017; Oudeyer et al., 2007) and information seeking (Schmidhuber et al., 1994; Gottlieb et al., 2013) abilities in an agent to encourage novel exploration (Lehman & Stanley, 2011).”
Curiosity-Driven Exploration by Self-Supervised Prediction (2017, background): “Many approaches measure information gain for exploration (Little & Sommer, 2014; Still & Precup, 2012; Storck et al., 1995).”
Automated Curriculum Learning for Neural Networks (2017, methodology): “Doing so brings us into contact with the intrinsic motivation literature (Barto, 2013), where various indicators of learning progress have been used as reward signals to encourage exploration, including compression progress (Schmidhuber, 1991), information acquisition (Storck et al., 1995), Bayesian”
Active Learning with Statistical Models (1996, methodology): “…1991), where we perform poorly (Linden & Weber, 1993), where we have low con(cid:12)dence (Thrun & M(cid:127)oller, 1992), where we expect it to change our model (Cohn, Atlas, & Ladner, 1990, 1994), and where we previously found data that resulted in learning (Schmidhuber & Storck, 1993).”
First Experiments with PowerPlay (2012, background): “Unlike our ﬁrst implementations of curious / creative / playful agents from the 1990s [17, 29, 18] ( cf .”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “Meta-learning was used in the context of reinforcement learning in 1995 [53], fol-”
Learning Awareness Models (2018, background): “…et al. (2016); Martius et al. (2013); Schmidhuber (2008); Mohamed & Rezende (2015); Haber et al. (2018b;a) Some of the ideas here, in particular the notion of choosing actions specifically to improve a model of the world, echo earlier speculative work of Schmidhuber (1991) and Storck et al. (1995).”
DORA The Explorer: Directed Outreaching Reinforcement Action-Selection (2018, methodology): “The idea of using reinforcement-learning techniques to estimate exploration can be traced back to Storck et al. (1995) and Meuleau & Bourgine (1999) who also analyzed propagation of uncertainties and exploration values.”
Some Considerations on Learning to Explore via Meta-Reinforcement Learning (2018, background): “As a result of these shortcomings, a variety of new approaches to exploration in deep RL have recently been suggested [34, 9, 31, 18, 4, 19, 17, 8, 23, 17, 8, 26, 25, 32, 33, 12, 24].”
Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning (2017, positive): “Conceptually and mathematically, our work is closest to prior work on curiosity and surprise [8, 19, 23, 24], although these works focus mainly on small finite MDPs.”
Stein Variational Policy Gradient (2017, methodology): “These techniques (Singh et al., 2004, Storck et al., 1995) attempt to train the agent with rewards beyond those given from the environment.”
Model-Based Active Exploration (2018, methodology): “By the 1990s, information gain was used as an intrinsic reward for reinforcement learning (RL) systems (Storck et al., 1995).”
Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments (2011, methodology): “Second, our result is theoretically sound and directly derived from first principles, in contrast to the more heuristic application [10] of traditional reinforcement learning to maximize the expected information gain.”