Transfer Learning via Inter-Task Mappings for Temporal Difference Learning

By Matthew E. Taylor, Peter Stone, Yaxin Liu. 2007

271 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

In Meta & Continual Learning

Citations in the corpus

Learning Parameterized Skills (2012, background): “is represented, for example, as a neural network, its weights and structure may be reused to execute similar tasks by assuming that a mapping between features of the source and target tasks is available [77].”
Disentangling Transfer in Continual Reinforcement Learning (2022, methodology): “[41, 50, 42, 39] present methods on how to ﬁnd and use mappings between different domains.”
Learning modular neural network policies for multi-task and multi-robot transfer (2016, background): “Past work in transfer on robotics domains includes shaping the target reward function from the source policy [18], [19] and learning a mapping between tasks [20].”
Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning (2017, background): “Taylor et al. (2007) manually construct a function to map a Q -function from one Markov decision process (MDP) to another.”
Reinforcement learning with multi-fidelity simulators (2014, methodology): “In TL, values or model parameters are typically used to bootstrap learning in the next task (e.g. [4]).”
Efficient reinforcement learning for robots using informative simulated priors (2015, background): “In contrast, we leverage nonparametric Gaussian processes (GPs) to incorporate data from simulators in a principled way.”
Learning Transferable Knowledge Through Embedding Spaces (2019, methodology): “This can be accomplished with distribution matching [164], inter-task mapping [178], or manifold alignment techniques [179; 180].”
Integrating reinforcement learning with human demonstrations of varying ability (2011, background): “However, many transfer methods restrict what type of learning algorithm is used by both agents (for instance, some methods require temporal difference learning [29] or a particular function approximator [32] to be used in both agents).”
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020, methodology): “As an example, in value function transfer (Taylor et al., 2007), the parameters of an action-value function qsource(s, a) learned in a source task are used to initialize the action-value function in the target task qtarget(s, a).”
Transfer in Reinforcement Learning via Shared Features (2012, methodology): “We use Keepaway to illustrate the use of learned shaping rewards on a standard but challenging benchmark that has been used in other transfer studies (Taylor et al., 2007).”
AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning (2021, background): “…an agent could use importance reweighting on samples (cid:104) s, a, r, s (cid:48) (cid:105) from sources (Tirinzoni et al., 2018; 2019) or start from the optimal source policy to initialize a learner in the target domain, as a near-optimal initializer (Taylor et al., 2007; Fernández et al., 2010).”
Using Bisimulation for Policy Transfer in MDPs (2010, background): “Taylor et al. (2007) transfer policies, represented as neural network action selectors, from a source to a target task.”
A comprehensive survey on safe reinforcement learning (2015, background): “These authors also learn an action-value function in a source task, translate the function into a target task via a handcoded inter-task mapping, and then use the transferred function to initialize the target task agent (Taylor et al., 2007).”
Half Field Offense: An Environment for Multiagent Learning and Ad Hoc Teamwork (2016, background): “Keepaway created interest, sparked research, and has served for many years as a testbed for new AI algorithms [19, 25, 26].”
Reinforcement Learning from Demonstration through Shaping (2015, methodology): “In the case of complex function approximators, where initialization is non-trivial or impossible, the Q-value reuse technique [Taylor et al., 2007] can be used to achieve the same effect as initialization.”
Transferring Instances for Model-Based Reinforcement Learning (2008, methodology): “In our previous wor k [20], we learn an action-value function for a source task, transla te the function into a target task via a hand-coded inter-task mapping , and then use the transferred function to initialize the target t ask agent.”
Transfer learning for reinforcement learning on a physical robot (2010, methodology): “Taylor et al. [19] explored Q-value reuse in temporal difference learning with good results.”