Task Transfer by Preference-Based Cost Learning
By Chi-Hung Hsu, Shu-Huan Chang, Da-Cheng Juan, Jia-Yu Pan, Yu-Ting Chen, Wei Wei, Shih-Chieh Chang. 2018
56 citations recorded by the source corpus.
The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task. Given their successes on robotic action planning, current methods mostly rely on two requirements: exactlyrelevant expert demonstrations or the explicitly-coded cost function on target task, both of which, however, are inconvenient to obtain in practice. In this paper, we relax these two strong conditions by developing a novel task transfer framework where the expert preference is applied as a guidance. In particular, we alternate the following two steps: Firstly, letting experts apply pre-defined preference rules to select related expert demonstrates for the target task. Secondly, based on the selection result, we learn the target cost function and trajectory distribution simultaneously via enhanced Adversarial MaxEnt IRL and generate more trajectories by the learned target distribution for the next preference selection. The theoretical analysis on the distribution learning and convergence of the proposed algorithm are provided. Extensive simulations on several benchmarks have been conducted for further verifying the effectiveness of the proposed method.
Subfields: Artificial Intelligence, Control and Systems Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Quoc V. Le
- Mingxing Tan
- Ruoming Pang
- Martin Wistuba
- Tejaswini Pedapati
- Ambrish Rawat
- Song Han
- Alan L. Yuille
In Control and Systems Engineering
Citations in the corpus
- A Survey on Neural Architecture Search (2019, methodology): “domain knowledge and the desired trade-o between objectives. Consequently, the multiobjective becomes a single-objective optimization problem which can be solved by any method discussed in Section 3. Hsu et al. (2018) propose to use the weighted sum as the aggregation function and use a reinforcemen”
- DATA: Differentiable ArchiTecture Approximation (2019, unknown): “, evolution-based NAS [13, 18, 26, 41, 43, 44, 45, 47], reinforcement learning-based NAS [2, 3, 21, 42, 56, 59, 60], and gradient-based NAS [11, 34, 35, 46, 53], which has achieved significant successes in a multitude of fields, including image classification [4, 12, 21, 30, 31, 34, 44, 53, 60], sem”
- Searching for Fast Model Families on Datacenter Accelerators (2021, methodology): “When designing fast models for inference with NAS, previous work employed multi-objective search [56, 18, 13, 29, 64, 27, 12, 21, 39, 16] to consider accuracy together with performance/efficiency.”