Deep Sketch-Guided Cartoon Video Inbetweening
By Xiaoyu Li, Bo Zhang, Jing Liao, Pedro V. Sander. 2021
34 citations recorded by the source corpus.
We propose a novel framework to produce cartoon videos by fetching the color information from two input keyframes while following the animated motion guided by a user sketch. The key idea of the proposed approach is to estimate the dense cross-domain correspondence between the sketch and cartoon video frames, and employ a blending module with occlusion estimation to synthesize the middle frame guided by the sketch. After that, the input frames and the synthetic frame equipped with established correspondence are fed into an arbitrary-time frame interpolation pipeline to generate and refine additional inbetween frames. Finally, a module to preserve temporal consistency is employed. Compared to common frame interpolation methods, our approach can address frames with relatively large motion and also has the flexibility to enable users to control the generated video sequences by editing the sketch guidance. By explicitly considering the correspondence between frames and the sketch, we can achieve higher quality results than other image synthesis methods. Our results show that our system generalizes well to different movie frames, achieving better results than existing solutions.
Subfields: Computer Vision and Pattern Recognition, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Aneeshan Sain
- Ayan Kumar Bhunia
- Tao A. Xiang
- Yi-Zhe Song
- Subhadeep Koley
- Viswanatha Reddy Gajjala
- Yongxin Yang
- Rohit Kundu
In Signal Processing
Citations in the corpus
- Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches (2022, background): “…12, 41, 46, 34, 53, 6, 2], sketch has also been employed in a variety of vision understanding tasks, including segmentation [22], video synthesis [28], representation learning [60, 3], object localisation [55], image-inpainting [62], 3D shape retrieval [31], 3D shape modelling [71], among others…”