VALAN: Vision and Language Agent Navigation
By Larry Lansing, Vihan Jain, Harsh Mehta, Haoshuo Huang, Eugene Ie. 2019
8 citations recorded by the source corpus.
VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation, in photo-realistic environments, such as Matterport3D and Google StreetView. We have added a minimal set of abstractions on top of SEED RL allowing us to generalize the architecture to solve a variety of other RL problems. In this article, we will describe VALAN's software abstraction and architecture, and also present an example of using VALAN to design agents for instruction-conditioned indoor navigation.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, LLM Agents & Tool Use, Multimodal Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Jason Baldridge
- Eugene Ie
- Alexander Ku
- Vihan Jain
- Peter Anderson
- Cordelia Schmid
- Mohit Bansal
- Yoav Artzi
In Natural Language Processing
- Jason Baldridge
- Eugene Ie
- Alexander Ku
- Vihan Jain
- Peter Anderson
- Yoav Artzi
- Roma Patel
- Piotr Wojciech Mirowski
In LLM Agents & Tool Use
In Multimodal Learning
- Eugene Ie
- Jason Baldridge
- Alexander Ku
- Vihan Jain
- Yoav Artzi
- Su Wang
- Mingde Zhao
- Piotr Wojciech Mirowski
Citations in the corpus
- On the Evaluation of Vision-and-Language Navigation Instructions (2021, methodology): “(2019) and implemented in VALAN (Lansing et al., 2019), which achieves a success rate (SR) of 45% on the R2R val-unseen split when trained on”
- Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding (2020, background): “Implementation Details Agents are implemented in VALAN (Lansing et al., 2019), a distributed reinforcement learning framework designed for VLN.”
- Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View (2020, methodology): “In our implementation, we use imitation learning on top of a scalable framework based on the Actor-Learner architecture (Lansing et al., 2019), instead of supervised learning using Hogwild!”