Gradient-free Policy Architecture Search and Adaptation
By Sayna Ebrahimi, Anna Rohrbach, Trevor Darrell. 2017
30 citations recorded by the source corpus.
We develop a method for policy architecture search and adaptation via gradient-free optimization which can learn to perform autonomous driving tasks. By learning from both demonstration and environmental reward we develop a model that can learn with relatively few early catastrophic failures. We first learn an architecture of appropriate complexity to perceive aspects of world state relevant to the expert demonstration, and then mitigate the effect of domain-shift during deployment by adapting a policy demonstrated in a source domain to rewards obtained in a target environment. We show that our approach allows safer learning than baseline methods, offering a reduced cumulative crash metric over the agent's lifetime as it learns to drive in a realistic simulated environment.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Trevor Darrell
- Sergey Levine
- Vladlen Koltun
- Luc Van Gool
- Pieter Abbeel
- Andreas Geiger
- Bernard Ghanem
- Fisher Yu
Citations in the corpus
- Driving Policy Transfer via Modularity and Abstraction (2018, background): “methods in the physical world requires expensive and time-consuming data collection. Reinforcement learning in particular is known for its high sample complexity and is conducted mainly in simulation [32,13,12]. Application of deep reinforcement learning to real vehicles has only been demonstrated i”
- Reinforcement Learning from Imperfect Demonstrations (2018, background): “Several previous efforts have shown that demonstrations can speed up RL by mimicking expert data with a temporal difference regularizer (Hester et al., 2017) or via gradient-free optimization (Ebrahimi et al., 2017), yet these methods presume near-optimal demonstrations.”
- Variational Adversarial Active Learning (2019, background): “The recent success of learning-based computer vision methods relies heavily on abundant annotated training examples, which may be prohibitively costly to label or impossible to obtain at large scale [10].”
- Conditional Affordance Learning for Driving in Urban Environments (2018, background): “ving simulators are often exploited in related work. Two popular driving simulators for research are the open source car racing game TORCS [2,23,26] and the commercial game Grand Theft Auto V (GTA V) [27,28]. However, TORCS is neither photorealistic nor complex enough, lacking important scene elemen”