Channel Attention Is All You Need for Video Frame Interpolation
By Myungsub Choi, Heewon Kim, Bohyung Han, Ning Xu, Kyoung Mu Lee. AAAI Conference on Artificial Intelligence, 2020
369 citations recorded by the source corpus.
Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion. To alleviate the limitation, we propose a simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component. Our algorithm employs a special feature reshaping operation, referred to as PixelShuffle, with a channel attention, which replaces the optical flow computation module. The main idea behind the design is to distribute the information in a feature map into multiple channels and extract motion information by attending the channels for pixel-level frame synthesis. The model given by this principle turns out to be effective in the presence of challenging motion and occlusion. We construct a comprehensive evaluation benchmark and demonstrate that the proposed approach achieves outstanding performance compared to the existing models with a component for optical flow computation.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/bb4a9650ca3946c70a7e92007cc61dc0dfd75522
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
Citations in the corpus
- Real-Time Intermediate Flow Estimation for Video Frame Interpolation (2020, methodology): “We provide a visual comparison of video clips with large motions from the Vimeo90K testing set in Figure 7, where SepConv [43] and DAIN [3] produce ghosting artifacts, and CAIN [10] causes missing-parts artifacts.”