Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View
By Harsh Mehta, Yoav Artzi, Jason Baldridge, Eugene Ie, Piotr Wojciech Mirowski. 2020
29 citations recorded by the source corpus.
The Touchdown dataset (Chen et al., 2019) provides instructions by human annotators for navigation through New York City streets and for resolving spatial descriptions at a given location. To enable the wider research community to work effectively with the Touchdown tasks, we are publicly releasing the 29k raw Street View panoramas needed for Touchdown. We follow the process used for the StreetLearn data release (Mirowski et al., 2019) to check panoramas for personally identifiable information and blur them as necessary. These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn. We also provide a reference implementation for both of the Touchdown tasks: vision and language navigation (VLN) and spatial description resolution (SDR). We compare our model results to those given in Chen et al. (2019) and show that the panoramas we have added to StreetLearn fully support both Touchdown tasks and can be used effectively for further research and comparison.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Jason Baldridge
- Eugene Ie
- Alexander Ku
- Vihan Jain
- Aniruddha Kembhavi
- Roozbeh Mottaghi
- Unnat Jain
- Kuo-Hao Zeng
Citations in the corpus
- On the Evaluation of Vision-and-Language Navigation Instructions (2021, methodology): “However, our findings are also relevant for similar datasets such as Touchdown (Chen et al., 2019; Mehta et al., 2020), CVDN (Thomason et al., 2019), REVERIE (Qi et al., 2020), and the multilingual Room-acrossRoom (RxR) dataset (Ku et al., 2020).”
- Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding (2020, background): “, 2020) and outdoor (Chen et al., 2019; Mehta et al., 2020) environments; success in these is based on clearly-defined, objective task completion rather than language or vision specific annotations.”
- AllenAct: A Framework for Embodied AI Research (2020, background): “…including visual navigation [12, 13, 14], question answering [15, 16], task completion [17], instruction following [18, 19], language grounding [20, 21], grasping [22, 23], object manipulation [24, 25], future prediction [26], and multi-agent collaboration [27, 28]; as well as using…”