Class-Based Styling: Real-Time Localized Style Transfer with Semantic Segmentation

By Lironne Kurzman, David Vázquez, Issam Hadj Laradji. 2019

25 citations recorded by the source corpus.

We propose a Class-Based Styling method (CBS) that can map different styles for different object classes in real-time. CBS achieves real-time performance by carrying out two steps simultaneously. While a semantic segmentation method is used to obtain the mask of each object class in a video frame, a styling method is used to style that frame globally. Then an object class can be styled by combining the segmentation mask and the styled image. The user can also select multiple styles so that different object classes can have different styles in a single frame. For semantic segmentation, we leverage DABNet that achieves high accuracy, yet only has 0.76 million parameters and runs at 104 FPS. For the style transfer step, we use the popular real-time method proposed by Johnson et al. [7]. We evaluated CBS on a video of the CityScapes dataset and observed high-quality localized style transfer results for different object classes and real-time performance.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Lawrence Carin
2. Jianqiao Li
3. Sijia Wang
4. Yulai Cong
5. Miaoyun Zhao
6. Chunyuan Li
7. Zhe Gan

Citations in the corpus

• GAN Memory with No Forgetting (2020, background): “Style-transfer techniques An extensive literature [23, 71, 79, 13, 62, 38] has explored how one can manipulate the style of an image (e.”