Forward Models: Supervised Learning with a Distal Teacher

By Michael I. Jordan, David E. Rumelhart. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Learning Continuous Control Policies by Stochastic Value Gradients (2015, background): “If a differentiable environment model is available, one can link together the policy, model, and reward function to compute an analytic policy gradient by backpropagation of reward along a trajectory [18, 11, 6, 9].”
Learning Parameterized Skills (2012, background): “This is similar in nature to the problem of learning inverse kinematic solutions by averaging examples on a non-convex set [29].”
Deep learning in neural networks: An overview (2014, methodology): “1 RL Through NN World Models Yields RNNs With Deep CAPs In the special case of an RL NN controller C interacting with a deterministic environment, a separate FNN called M can learn to become C’s world model through system identification, predicting C’s inputs from previous actions and inputs (e.g., ”
Curiosity-Driven Exploration by Self-Supervised Prediction (2017, methodology): “Our approach of jointly training forward and inverse models for learning a feature space has similarities to (Agrawal et al., 2016; Jordan & Rumelhart, 1992; Wolpert et al., 1995), but these works use the learned models of dynamics for planning a sequence of actions instead of exploration.”
Model-Ensemble Trust-Region Policy Optimization (2018, neutral): “Although many previous studies have shown promising results on relatively simple domains (Nguyen & Widrow, 1990; Schmidhuber & Huber, 1991; Jordan & Rumelhart, 1992; Gal et al., 2016), so far their applications on more challenging domains have either required a combination with model-free…”
Learning Visual Predictive Models of Physics for Playing Billiards (2015, background): “(Jordan & Rumelhart, 1992; Wolpert et al., 1995; Haruno et al., 2001; Todorov & Ghahramani, 2003) proposed using internal models of the external world for planning actions.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The first combination of an RL RNN C and an UL RNN M was ours and dates back to 1990 [223, 222, 226, 227], generalizing earlier similar controller/model systems (CM systems) based on FNNs [298, 179]; compare related work [177, 119, 301, 300, 209, 120, 178, 302, 73, 45, 144, 166, 153, 196, 60][245, S”
Optimal Ordered Problem Solver (2002, methodology): “The method was inspired by previous combinations of non recurrent, reactive world models and controllers (Werbos, 1987, Nguyen and Widrow, 1989, Jordan and Rumelhart, 1990).”
Zero-Shot Visual Imitation (2018, background): “The works of Agrawal et al. (2016); Jordan & Rumelhart (1992); Pathak et al. (2017); Wolpert et al. (1995) jointly learn forward and inverse dynamics model but do not optimize for consistency between the forward and inverse dynamics.”
Goal Babbling Permits Direct Learning of Inverse Kinematics (2010, methodology): “Several learning schemes to ﬁnd appropriate joint angles have been proposed including feedback-based learning schemes (e.g., [11]) which resemble Jacobian-based controllers [12], associative procedures (e.g., [13]) as well as feed-forward-based schemes(e.g.,[14]).”
Computational principles of sensorimotor control that minimize uncertainty and variability (2007, background): “2 A ) include a forward model: an internal representation of the environment that is used to transform the efference copysignalintoasensoryprediction(Jordan&Rumelhart, 1992;Wolpert&Miall,1996;Davidson&Wolpert,2005).”
Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. (2004, methodology): “For example, the demands of planning appear to require some means of forecasting the outcome of one’s actions, suggesting the addition of a forward model to the present architecture (Jordan & Rumelhart, 1992; see also Elsner & Hommel, 2001).”
Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability (2024, unknown): “After synthesizing the video plan, we can convert the resultant image trajectory to executable actions by a non-causal inverse dynamics model [4, 65, 71], which can be efficiently learned from much fewer data compared with the imitation learning pipeline [2, 9].”
Understanding normal and impaired word reading: computational principles in quasi-regular domains. (1996, methodology): “Such details include the specific orthographic and phonological representation schemes, the logarithmic frequency compression used in training, the use of error scores to model naming latencies, and the use of a supervised, error-correcting training procedure (but see Jordan & Rumelhart, 1992).”
Learning Forward Models for the Operational Space Control of Redundant Robots (2010, background): “For a general review on learning non-linear models, readers can refer to Jordan and Rumelhart (1992).”