Learning Object-conditioned Exploration using Distributed Soft Actor Critic

By Ayzaan Wahid, Austin Stone, Kevin Chen, Brian Ichter, Alexander Toshev. 2020

23 citations recorded by the source corpus.

Object navigation is defined as navigating to an object of a given label in a complex, unexplored environment. In its general form, this problem poses several challenges for Robotics: semantic exploration of unknown environments in search of an object and low-level control. In this work we study object-guided exploration and low-level control, and present an end-to-end trained navigation policy achieving a success rate of 0.68 and SPL of 0.58 on unseen, visually complex scans of real homes. We propose a highly scalable implementation of an off-policy Reinforcement Learning algorithm, distributed Soft Actor Critic, which allows the system to utilize 98M experience steps in 24 hours on 8 GPUs. Our system learns to control a differential drive mobile base in simulation from a stack of high dimensional observations commonly used on robotic platforms. The learned policy is capable of object-guided exploratory behaviors and low-level control learned from pure experiences in realistic environments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Silvio Savarese
2. Song-Chun Zhu
3. Siyuan Huang
4. Fei Xia
5. Yixin Zhu
6. Brian Ichter
7. Baoxiong Jia
8. Kevin Chen

In Computer Vision and Pattern Recognition

1. Silvio Savarese
2. Luc Van Gool
3. Jitendra Malik
4. Li Fei-Fei
5. Song-Chun Zhu
6. Kristen Grauman
7. Dacheng Tao
8. Andrew Zisserman

Citations in the corpus

• A Survey of Embodied AI: From Simulators to Research Tasks (2021, neutral): “Object Navigation is one of the most straightforward tasks, yet one of the most challenging tasks in embodied AI.”
• Open-vocabulary Queryable Scene Representations for Real World Planning (2022, background): “Other algorithms [24]–[29] do not require a map and can decide where to go based directly on the current observations and memories, without maintaining a global representation of the environment.”
• Diffusion-based Generation, Optimization, and Planning in 3D Scenes (2023, methodology): “However, existing work relies heavily on model-based planners with the single-step dynamic model [5,11,48,67,74,80], lacking a trajectory-level optimization for long-horizon planning.”
• Topological Planning with Transformers for Vision-and-Language Navigation (2020, background): “Such approaches often require millions of frames of experience [11, 50, 52], and yet performance substantially degrades without ground truth odometry [11, 52].”