Joint embeddings of shapes and images via CNN image purification

By Yangyan Li, Hao Su, Charles Ruizhongtai Qi, Noa Fish, Daniel Cohen-Or, Leonidas J. Guibas. 2015

207 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Learning a Predictable and Generative Vector Representation for Objects (2016, methodology): “Many approaches achieve this via alignment [21,22,23]; the most similar work is Li et al. [24], which learns a joint, but non-generative, embedding of images and 3D shapes.”
Deep Metric Learning via Lifted Structured Feature Embedding (2015, methodology): “…a stochastic gradient descent by drawing pairs or triplets of points uniformly at random [14, 1, 22], our approach deviates from those methods in two ways: (1) it biases the sample towards including “difficult” pairs, just like a subgradient of Ji,j would use the close negative pairs 1; (2) it…”
Single Image 3D Interpreter Network (2016, background): “The other is to train on synthetically rendered images of 3D objects [10,11], where complete 3D structure is available.”
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views (2015, background): “Furthermore, 2D images and 3D shapes/scans can be connected through our pipeline [18], thus information can be transported between the two domains bidirectionally.”
Marr Revisited: 2D-3D Alignment via Surface Normal Prediction (2016, background): “veloped, such as learning a mapping from CNN features to a 3D light-field embedding space for view-invariant shape retrieval [25] and retrieval using AlexNet [22] pool5 features [2].”
VSE++: Improving Visual-Semantic Embeddings with Hard Negatives (2017, background): “Examples include shape-image embeddings ([20]) for shape inference, bilingual word embeddings ([38]), human pose-image embeddings for 3D pose inference ([19]), fine-grained recognition ([25]), zero-shot learning ([9]), and modality conversion via synthesis ([25, 26]).”
3D Face Reconstruction by Learning from Synthetic Data (2016, methodology): “Finally, as an optional step, we demonstrate how the resulting facial structure can be further reﬁned by a shape from shading algorithm in Section 7.”
Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve (2020, methodology): “This stands in contrast to the embedding space construction of [25] which explicitly enforces shape similarity in its light ﬁeld descriptors. inference time, however, we can still embed new 3D models into the space without training, by using our trained model to compute their feature embeddings.”
Transfer of View-manifold Learning to Similarity Perception of Novel Objects (2017, methodology): “From a technical point of view, our OPnet performs better than earlier approaches (Li et al., 2015) in instance and categorical retrieval of novel objects.”
Deep representation of industrial components using simulated images (2017, methodology): “The fine-tuned CNN with mixed data creates an embedded feature space [19].”
DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding (2016, background): “a, graphics for vision, or synthesis for analysis), these efforts focus on mostly object rendering, either in color [32, 22] or depth [30, 11].”
Learning Shape Templates With Structured Implicit Functions (2019, methodology): “a similar technique to CNN purification [28] or network distillation [16] and train a second network that regresses from RGB to the template parameters already found through our 3D-to-3D training scheme.”
Deep Exemplar 2D-3D Detection by Adapting from Real to Rendered Views (2015, background): “Despite recent progress on 2D-3D matching and retrieval [27, 35, 48], detection by 2D-3D alignment lags behind state-of-the-art object detection systems based on annotated images, e.”
What Do Single-View 3D Reconstruction Networks Learn? (2019, background): “[25], which learns to embed images and shapes in a joint space.”
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene (2017, methodology): “Similarly, in CAD retrieval scenarios, one assumes the object [3, 4, 13, 19, 20] or scene [17] can be represented in terms of a pre-determined dictionary of shapes; a great deal of earlier work [12, 18, 27] tackled this with a dictionary of box models.”