Auxiliary Tasks Speed Up Learning PointGoal Navigation
By Joel Ye, Dhruv Batra, Erik Wijmans, Abhishek Das. 2020
87 citations recorded by the source corpus.
PointGoal Navigation is an embodied task that requires agents to navigate to a specified point in an unseen environment. Wijmans et al. showed that this task is solvable but their method is computationally prohibitive, requiring 2.5 billion frames and 180 GPU-days. In this work, we develop a method to significantly increase sample and time efficiency in learning PointNav using self-supervised auxiliary tasks (e.g. predicting the action taken between two egocentric observations, predicting the distance between two observations from a trajectory,etc.).We find that naively combining multiple auxiliary tasks improves sample efficiency,but only provides marginal gains beyond a point. To overcome this, we use attention to combine representations learnt from individual auxiliary tasks. Our best agent is 5x faster to reach the performance of the previous state-of-the-art, DD-PPO, at 40M frames, and improves on DD-PPO's performance at40M frames by 0.16 SPL. Our code is publicly available at this https URL.
Subfields: Reinforcement Learning, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Reinforcement Learning
- Dhruv Batra
- Erik Wijmans
- Devendra Singh Chaplot
- Yilun Du
- Manolis Savva
- Jitendra Malik
- Chuang Gan
- Vladlen Koltun
In Robotics & Control
- Dhruv Batra
- Erik Wijmans
- Manolis Savva
- Vladlen Koltun
- Devendra Singh Chaplot
- Jitendra Malik
- Chuang Gan
- Yilun Du
Citations in the corpus
- Habitat 2.0: Training Home Assistants to Rearrange their Habitat (2021, background): “[13] Joel Ye, Dhruv Batra, Erik Wijmans, and Abhishek Das.”
- A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “with a GPS and compass that allows it to access to their location coordinates, and implicitly their orientation relative to the goal position [23, 80].”
- Tool Learning with Foundation Models (2023, unknown): “Tasks typically assigned to agents include exploration (Ramakrishnan et al., 2021), navigation (Ye et al., 2021), question answering (Yu et al., 2019) within the simulated environment, or more interactive embodied task (Abramson et al., 2022) based on human instructions.”
- Curious Representation Learning for Embodied Intelligence (2021, methodology): “Perhaps most related to our work is the work of Ye [49], who show that using auxiliary tasks can improve PointGoal navigation results in the Gibson environment [47].”
- How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget (2020, background): “agent variants for a fixed number of samples – 75 million steps, a high but feasible number of samples [5, 10, 21, 22, 31].”