Learning to compare image patches via convolutional neural networks

By Sergey Zagoruyko, Nikos Komodakis. 2015

1,483 citations recorded by the source corpus.

In this paper we show how to learn directly from image data (i.e., without resorting to manually-designed features) a general similarity function for comparing image patches, which is a task of fundamental importance for many computer vision problems. To encode such a function, we opt for a CNN-based model that is trained to account for a wide variety of changes in image appearance. To that end, we explore and study multiple neural network architectures, which are specifically adapted to this task. We show that such an approach can significantly outperform the state-of-the-art on several problems and benchmark datasets.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Learning Spread-Out Local Feature Descriptors (2017, methodology): “4Subscript “2str” means central-surround network proposed in [27].”
Set2Graph: Learning Graphs From Sets (2020, methodology): “…learning Previous models suggested for learning pairwise relations in sets were mostly of the form H 2 ( X ) = m ( φ ( x i ) , φ ( x j )) (Hsu et al., 2017; Bromley et al., 1994; Simo-Serra et al., 2015; Zagoruyko & Komodakis, 2015; Bell & Bala, 2015; Ahmed et al., 2015; Vo & Hays, 2016).”
Learning to cluster in order to Transfer across domains and tasks (2017, methodology): “However, the Siamese architecture used in Zagoruyko & Komodakis (2015) is not efﬁcient in both training and inference, especially when pairwise information is dense.”
Learning to Compare: Relation Network for Few-Shot Learning (2017, background): “In contrast, our framework further defines a relation classifier CNN, in the style of [33, 44, 14] (While [33] focuses on reasoning about relation between two objects in a same image which is to address a different problem.”
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials (2018, background): “CNNs have been proven successful for learning similarity measures between two or more image patches for stereo [20, 40, 41] as well as multi-view stereo [14].”
Semi-parametric Topological Memory for Navigation (2018, methodology): “We use a siamese architecture for the network R , akin to Zagoruyko & Komodakis (2015).”
Fully-Convolutional Siamese Networks for Object Tracking (2016, methodology): “Deep Siamese conv-nets have previously been applied to tasks such as face verification [14,18,20], keypoint descriptor learning [19,21] and one-shot character recognition [22].”
Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks (2017, methodology): “Similarly to the experimental setup used in [16], we train the network on matching and non-matching pairs.”
Rotation Equivariant Vector Field Networks (2016, methodology): “We have presented a new way of hard-coding rotation equivariance in CNNs by applying each ﬁlter at different orientations and extracting a vector ﬁeld feature map that encodes the maximum activation magnitude and angle.”
Dense Human Body Correspondences Using Convolutional Networks (2015, background): “Several recent papers [27, 15, 49, 14] have successfully used this idea for finding correspondences between 2D images, and have shown that descriptors learned from deep neural networks are significantly better than generic pixel-wise descriptors in this context.”
Pixel-Level Domain Transfer (2016, background): “Several architecture families have been proposed to feed a pair of images to compare them but a simple stack across the channel axis has shown the best performance as studied in [26].”
CliqueCNN: Deep Unsupervised Exemplar Learning (2016, background): “vision community, leading to the formulation as a ranking [34], regression [11], and classiﬁcation [28] task. With the recent advances of convolutional neural networks (CNN), two-stream architectures [36] and ranking losses [32] have shown great improvements. However, to achieve their performance ga”
Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches (2015, neutral): “, 2014), and convolutional neural networks (Zagoruyko and Komodakis, 2015; Han et al., 2015).”
Generic 3D Representation via Pose Estimation and Matching (2016, background): “Therefore, we presume (and empirically observe) that the generality of our 3D representation is mostly attributed to the camera pose estimation component. scribing, and matching image features, either through a handcrafting the feature [35,11,39,38,5,40] or learning it [50,25,65,9,66,49,19].”
Learned Multi-patch Similarity (2017, background): “An extensive study of similarity measures based on different CNN architectures is presented in [26].”
Graph Matching Networks for Learning the Similarity of Graph Structured Objects (2019, background): “Siamese networks have achieved great success in many visual recognition and veriﬁcation tasks (Brom-ley et al., 1994; Baldi & Chauvin, 1993; Koch et al., 2015; Bertinetto et al., 2016; Zagoruyko & Komodakis, 2015).”
The Visual Object Tracking VOT2017 Challenge Results (2017, methodology): “Then, by using the DeepCompare [80] the system adaptively modifies the object bounding box position and shape.”
Neural network-based clustering using pairwise constraints (2015, background): “…learning (Chopra et al., 2005), dimensionality reduction (Hadsell et al., 2006), semi-supervised embedding (Weston et al., 2008), and some applications to image data, such as in learning to match patches (Han et al., 2015; Zagoruyko & Komodakis, 2015) and feature points (Simo-Serra et al., 2015).”