CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration
By S. Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, Shuran Song. 2022
118 citations recorded by the source corpus.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Chelsea Finn
- Ali Farhadi
- Mohit Bansal
- Trevor Darrell
- Li Fei-Fei
- Pieter Abbeel
- Quoc V. Le
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Trevor Darrell
- Jitendra Malik
- Dacheng Tao
- Li Fei-Fei
- Kristen Grauman
- Xiaogang Wang
- Abhinav Kumar Gupta
In Multimodal Learning
- Chuang Gan
- Ludwig Schmidt
- Ali Farhadi
- Aniruddha Kembhavi
- Srivatsa Kundurthy
- Antonio Torralba
- Josh Tenenbaum
- Yining Hong
In Robotics & Control
- Sergey Levine
- Chelsea Finn
- Roozbeh Mottaghi
- Andy Zeng
- Pierre Sermanet
- Tianhe Yu
- Luca Weihs
- Brian Ichter
Citations in the corpus
- LAION-5B: An open large-scale dataset for training next generation image-text models (2022, methodology): “[17] Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song.”
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (2023, methodology): “Prior works have studied the use of VLMs for robotics [48, 61, 49, 17, 70, 71, 72], and form part of the inspiration for this work.”
- Open-vocabulary Queryable Scene Representations for Real World Planning (2022, methodology): “Recently, methods that leverage pre-trained image-text models can do zero-shot Object Goal Navigation [31], [32].”
- Visual Language Maps for Robot Navigation (2022, methodology): “, CoW [12] and LM-Nav [13], and, in particular,”
- 3D-LLM: Injecting the 3D World into Large Language Models (2023, methodology): “On the contrary, numerous methods have been proposed to extract 3D features from 2D multi-view images [24, 20, 17, 21].”
- ConceptFusion: Open-set Multimodal 3D Mapping (2023, methodology): “Approaches such as VL-Maps [44], NLMap-SayCan [47], CoWs [46, 65], CLIP-Fields [43] leverage LSeg [24]; while OpenScene [48] experiments with both LSeg [24] and OpenSeg [18].”
- ChatGPT for Robotics: Design Principles and Model Abilities (2023, background): “array([30,255,255]) # Threshold the image to get only the orange color mask = cv2.”
- Your Diffusion Model is Secretly a Zero-Shot Classifier (2023, unknown): “Since then there has been a surge towards building a new category of classifiers, known as zero-shot or open-vocabulary clas-sifiers, that are capable of detecting a wide range of class categories [25, 47, 48, 1].”
- ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning (2023, background): “…the 2D representations produced by image and language foundation models to the 3D world and show impressive results on open-vocabulary tasks, including language-guided object grounding [17], [18], [24], [26], [36], 3D reasoning [37], [38], robot manipulation [39]–[41] and navigation [42], [43].”
- ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (2022, background): “10 [128] Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song.”
- OpenMask3D: Open-Vocabulary 3D Instance Segmentation (2023, background): “Recent success of 2D open-vocabulary segmentation models such as LSeg [39], OpenSeg [18], and OV-Seg[41] has motivated researchers in the field of 3D scene understanding to explore the open vocabulary setting [6, 10, 17, 21, 27, 29, 32, 34, 45, 49, 56, 57].”
- ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings (2022, methodology): “The SPL of our approach is 1.5% lower than CoW.”