A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation

By Dinh C. Nguyen, Binh-Son Hua, Lap-Fai Craig Yu, Sai-Kit Yeung. 2016

46 citations recorded by the source corpus.

Recent advances of 3D acquisition devices have enabled large-scale acquisition of 3D scene data. Such data, if completely and well annotated, can serve as useful ingredients for a wide spectrum of computer vision and graphics works such as data-driven modeling and scene understanding, object detection and recognition. However, annotating a vast amount of 3D scene data remains challenging due to the lack of an effective tool and/or the complexity of 3D scenes (e.g., clutter, varying illumination conditions). This paper aims to build a robust annotation tool that effectively and conveniently enables the segmentation and annotation of massive 3D data. Our tool works by coupling 2D and 3D information via an interactive framework, through which users can provide high-level semantic annotation for objects. We have experimented our tool and found that a typical indoor scene could be well segmented and annotated in less than 30 minutes by using the tool, as opposed to a few hours if done manually. Along with the tool, we created a dataset of over a hundred 3D scenes associated with complete annotations using our tool. Both the tool and dataset are available at http://scenenn.net.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding (2020, methodology): “nd is therefore directly applicable in our setting. Interactive Annotation Tools for 3D Scenes Our interactive tool is similar in spirit to existing tools for annotating reconstructed triangle meshes [17,50,67] and synthetic images from video games [7,52]. All of these tools, including ours, leverag”
Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, background): “A potential treatment is to remove such background using human annotators [28].”
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes (2017, background): “The semantic seg-mentations are more than an order of magnitude larger than any previous RGB-D dataset.”
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera (2019, background): “[37] has a similar starting point; the resulting over-segments are further grouped into larger regions based on geometry and appearance cues.”
SceneNN: A Scene Meshes Dataset with aNNotations (2016, methodology): “More technical details of our annotation tool could be found in the technical report [32].”
Pointwise Convolutional Neural Networks (2017, methodology): “In the 3D domain, interactive semantic segmentation [40, 39] relied on user strokes to propagate segmentation.”