Neural best-buddies

By Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Baoquan Chen, Daniel Cohen-Or. 2018

43 citations recorded by the source corpus.

Correspondence between images is a fundamental problem in computer vision, with a variety of graphics applications. This paper presents a novel method for sparse cross-domain correspondence. Our method is designed for pairs of images where the main objects of interest may belong to different semantic categories and differ drastically in shape and appearance, yet still contain semantically related or geometrically similar parts. Our approach operates on hierarchies of deep features, extracted from the input images by a pre-trained CNN. Specifically, starting from the coarsest layer in both hierarchies, we search for Neural Best Buddies (NBB): pairs of neurons that are mutual nearest neighbors. The key idea is then to percolate NBBs through the hierarchy, while narrowing down the search regions at each level and retaining only NBBs with significant activations. Furthermore, in order to overcome differences in appearance, each pair of search regions is transformed into a common appearance. We evaluate our method via a user study, in addition to comparisons with alternative correspondence approaches. The usefulness of our method is demonstrated using a variety of graphics applications, including cross-domain image alignment, creation of hybrid images, automatic image morphing, and more.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Style Transfer by Relaxed Optimal Transport and Self-Similarity (2019, methodology): “Distinct from the framework of Neural Style Transfer, there are several recent methods [17, 1] that use similarities between deep neural features to build a correspondence map between the content image and style image, and warp the style image onto the content image.”
Cross-Domain Correspondence Learning for Exemplar-Based Image Translation (2020, unknown): “Following works further improve the correspondence quality by incorporating additional annotations [51, 7, 11, 12, 21, 25], adopting coarse-to-fine strategy [27] or retaining reliable sparse matchings [1].”