Learning to generate sub-goals for action sequences

By J. Urgen Schmidhuber. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “To allow for memories of previous events in partially observable worlds (Section 6.3), the most general variant of this technique uses RNNs instead of FNNs to implement bothM and C (Feldkamp & Puskorius, 1998; Schmidhuber, 1990d, 1991c).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “In particular, gradient-based subgoal discovery with RNNs decomposes RL tasks into subtasks for submodules [225].”
One Big Net For Everything (2018, background): “Also in 1990, gradient-based recurrent subgoal generators [57, 58, 80] used special start and goal-deﬁning input vectors, also for an evaluator network predicting the costs and rewards associated with moving from starts to goals.”
Hindsight policy gradients (2017, background): “An interesting application for goal-conditional policies is hierarchical reinforcement learning [12–30], where they may help in deﬁning options for arbitrary subgoals [22].”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, background): “…representation of the current look-ahead time), desire ( t ) ∈ R n (a unique representation of the desired cumulative reward to be achieved until the end of the current look-ahead time), and extra ( t ) ∈ R q to encode additional user-given goals (as we have done since 1990, e.g., [44, 55, 51]).”
Improving Generalization in Meta Reinforcement Learning using Learned Objectives (2019, background): “In turn, these could create pressure to adapt the policy’s actions to explore unknown dynamics in the environment (Schmidhuber, 1991b; 1990; Houthooft et al., 2016; Pathak et al., 2017).”
HQ-Learning (1997, methodology): “%) does decrease performance, but much more noise does not lead to much worseresults.4 Previous WorkOther authors proposed hierarchical reinforcement learning techniques, e.g., Schmidhuber (1991b),Dayan and Hinton (1993), Moore and Atkeson (1993), Tham (1995), Sutton (1995).”
Planning simple trajectories using neural subgoal generators (1993, background): “Simpletrajectory planning tasks are solved with the help of `neural' gradient-based algorithms forlearning without a teacher to generate sequences of appropriate subgoals in response to novelstart/goal combinations.”