Transfer of View-manifold Learning to Similarity Perception of Novel Objects

By Xingyu Lin, Yimeng Zhang, Alan L. Yuille, Tai Sing Lee. 2017

13 citations recorded by the source corpus.

We develop a model of perceptual similarity judgment based on re-training a deep convolution neural network (DCNN) that learns to associate different views of each 3D object to capture the notion of object persistence and continuity in our visual experience. The re-training process effectively performs distance metric learning under the object persistency constraints, to modify the view-manifold of object representations. It reduces the effective distance between the representations of different views of the same object without compromising the distance between those of the views of different objects, resulting in the untangling of the view-manifolds between individual objects within the same category and across categories. This untangling enables the model to discriminate and recognize objects within the same category, independent of viewpoints. We found that this ability is not limited to the trained objects, but transfers to novel objects in both trained and untrained categories, as well as to a variety of completely novel artificial synthetic objects. This transfer in learning suggests the modification of distance metrics in view- manifolds is more general and abstract, likely at the levels of parts, and independent of the specific objects or categories experienced during training. Interestingly, the resulting transformation of feature representation in the deep networks is found to significantly better match human perceptual similarity judgment than AlexNet, suggesting that object persistence could be an important constraint in the development of perceptual similarity judgment in biological neural networks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Few-Shot Image Recognition by Predicting Parameters from Activations (2017, methodology): “The other method we compare is Siamese-Triplet Network [14, 22].”
Deep Nets: What have They Ever Done for Vision? (2018, methodology): “For example, Lin et al. (2017) trained a Siamese network to learn similarity for objects in ShapeNet (Chang et al., 2015) and then this similarity measure was used to cluster objects in the Tufa dataset (Salakhutdinov et al., 2012).”
Adaptive Masked Weight Imprinting for Few-Shot Segmentation (2019, background): “Most of the few-shot learning literature focused on image classiﬁcation (Finn et al., 2017)(Koch et al., 2015)(Lin et al., 2017)(Vinyals et al., 2016)(Snell et al., 2017) (Qi et al., 2017)(Qiao et al., 2017).”