Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation
By Yu Xiang, Christopher Xie, Arsalan Mousavian, Dieter Fox. 2020
115 citations recorded by the source corpus.
Segmenting unseen objects in cluttered scenes is an important skill that robots need to acquire in order to perform tasks in new environments. In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data. A metric learning loss function is utilized to learn to produce pixel-wise feature embeddings such that pixels from the same object are close to each other and pixels from different objects are separated in the embedding space. With the learned feature embeddings, a mean shift clustering algorithm can be applied to discover and segment unseen objects. We further improve the segmentation accuracy with a new two-stage clustering algorithm. Our method demonstrates that non-photorealistic synthetic RGB and depth images can be used to learn feature embeddings that transfer well to real-world images for unseen object instance segmentation.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Dieter Fox
- Rudolph Triebel
- Arsalan Mousavian
- Martin Sundermeyer
- Mohit Shridhar
- Anelia Angelova
- Chris Paxton
- Luke S. Zettlemoyer
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Trevor Darrell
- Jitendra Malik
- Dieter Fox
- Dacheng Tao
- Cordelia Schmid
- Kristen Grauman
- Xiaogang Wang
In 3D Vision & Pose
- Arsalan Mousavian
- Chris Paxton
- Mohit Shridhar
- Yonatan Bisk
- Jesse Thomason
- Dieter Fox
- Anelia Angelova
- In-So Kweon
Citations in the corpus
- Learning Open-World Object Proposals without Learning to Classify (2021, background): “The objectness is often trained on synthetic RGB and depth data [38], [39] or on”
- Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes (2021, methodology): “Full Inference Pipeline: We segment unknown objects from an RGB-D image using [15].”
- Language Grounding with 3D Objects (2021, methodology): “We used unknown object instance segmentation [48] to segment out each object.”