Avatar-Net: Multi-scale Zero-Shot Style Transfer by Feature Decoration

By Lu Sheng, Ziyi Lin, Jing Shao, Xiaogang Wang. 2018

325 citations recorded by the source corpus.

Zero-shot artistic style transfer is an important image synthesis problem aiming at transferring arbitrary style into content images. However, the trade-off between the generalization and efficiency in existing methods impedes a high quality zero-shot style transfer in real-time. In this paper, we resolve this dilemma and propose an efficient yet effective Avatar-Net that enables visually plausible multi-scale transfer for arbitrary style. The key ingredient of our method is a style decorator that makes up the content features by semantically aligned style features from an arbitrary style image, which does not only holistically match their feature distributions but also preserve detailed style patterns in the decorated features. By embedding this module into an image reconstruction network that fuses multi-scale style abstractions, the Avatar-Net renders multi-scale stylization for any style image in one feed-forward pass. We demonstrate the state-of-the-art effectiveness and efficiency of the proposed method in generating high-quality stylized images, with a series of successive applications include multiple style integration, video stylization and etc.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

StyTr2: Image Style Transfer with Transformers (2021, methodology): “We compare our method with AdaIN [22], Avater [49], SANet [39], AAMS [63], MAST [14], MCC [13], ArtFlow [1], AdaAttN [35], IEST [9], and StyleFormer [56].”
Arbitrary Style Transfer With Style-Attentional Networks (2018, methodology): “For arbitrary style transfer, a few methods [13, 7, 20] holistically adjust the content features to match the secondorder statistics of the style features.”
Photorealistic Style Transfer via Wavelet Transforms (2019, background): “[8], many artistic style transfer studies have been proposed to synthesize stylized images through either iterative optimization [9], finding dense correspondence [20, 28, 10] or manipulating features in pre-trained networks [13, 18].”
Dynamic Instance Normalization for Arbitrary Style Transfer (2019, background): “Following their work, (Sheng et al. 2018) further extends AdaIN to multiscale stylization for better visual quality.”
Infrared and Visible Image Fusion with ResNet and zero-phase component analysis (2018, methodology): “[27] also use ZCA operation in their style transfer method.”
Normalization Techniques in Training DNNs: Methodology, Analysis and Application (2020, methodology): “Some methods [246] also seek to provide a good trade-off between AdaIN (which enjoys higher computational efficiency) and WCT (which synthesizes images visually closer to a given style).”
PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training (2021, methodology): “To perform the target-to-source stylization, we use the zero-shot style transfer model from [45] (AvatarNet).”
AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer (2021, methodology): “Avatar-Net [32] utilizes AdaIN for multi-scale transferring and introduces style decorators with patch matching strategies, which results in blur stylization results with obvious plaques (1st, 6th and 8th rows).”
StyleFormer: Real-time Arbitrary Style Transfer via Parametric Style Composition (2021, methodology): “The third line of works [24, 25, 28, 29, 30] tried to swap the most relevant style patches to match the content patches at the feature level, which may lead to flaky results.”