ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes

By Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, Leonidas J. Guibas. European Conference on Computer Vision, 2020

487 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/53794499a3830c3ebb365ecc57f0e8c8a20a682d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, background): “Some pioneering works have already begun in this direction (Achlioptas et al., 2020; Chen et al., 2020) and we anticipate the trend to shift more towards developing algorithms for understanding as well as the generation of 3D scenes, while utilizing language as a main or auxiliary modality.”
An Embodied Generalist Agent in 3D World (2023, unknown): “…descriptions (Luo et al., 2023); 2) object-in-the-scene captions , where the goal is to generate the referring expressions of objects in a 3D scene context (Achlioptas et al., 2020; Zhu et al., 2023c); and 3) scene-level captions , which focuses on depicting global 3D scene using natural language.”
CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation (2021, background): “More recently, meth-ods that use text to localize objects in 3D scene have been explored [2,6].”
A Survey of Visual Transformers (2021, methodology): “MVT achieves the SoTA performance on Nr3D and Sr3D datasets [205].”
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language (2019, background): “[2] introduces a new dataset and task that focuses on disambiguating objects from the same category with known localizations.”
Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, unknown): “Embodied interaction includes not only question-answering, but also performing tasks like grasping and placing objects based on human instructions.”
ScanQA: 3D Question Answering for Spatial Scene Understanding (2021, unknown): “Currently, 3D spatial-understanding models can be developed for the 3D object localization task of ScanRe-fer [10], the dialog-based localization of ReferIt3D [1], and the 3D object captioning task of Scan2Cap [13].”
3D-LLM: Injecting the 3D World into Large Language Models (2023, background): “Another line of research that is similar to ours is 3D and language [5, 45, 8, 20, 1, 16, 22, 45, 3].”
Decomposing NeRF for Editing via Feature Field Distillation (2022, methodology): “Furthermore, while this work focuses on editable view synthesis, it is also intriguing to transfer DFF to other applications, including 3D registration of text queries [16, 1, 47, 4] or robot teaching [32, 80].”
SQA3D: Situated Question Answering in 3D Scenes (2022, methodology): “Further, we hypothesize the bottleneck could also be on the 3D captions, as the results verify the consistent impact on model performances brought by a different source of captions (ScanRefer → ReferIt3D).”