A convolutional neural network approach for half-pel interpolation in video coding

By Ning Yan, Dong Liu, Houqiang Li, Feng Wu. 2017

49 citations recorded by the source corpus.

Motion compensation is a fundamental technology in video coding to remove the temporal redundancy between video frames. To further improve the coding efficiency, sub-pel motion compensation has been utilized, which requires interpolation of fractional samples. The video coding standards usually adopt fixed interpolation filters that are derived from the signal processing theory. However, as video signal is not stationary, the fixed interpolation filters may turn out less efficient. Inspired by the great success of convolutional neural network (CNN) in computer vision, we propose to design a CNN-based interpolation filter (CNNIF) for video coding. Different from previous studies, one difficulty for training CNNIF is the lack of ground-truth since the fractional samples are actually not available. Our solution for this problem is to derive the “ground-truth” of fractional samples by smoothing high-resolution images, which is verified to be effective by the conducted experiments. Compared to the fixed half-pel interpolation filter for luma in High Efficiency Video Coding (HEVC), our proposed CNNIF achieves up to 3.2% and on average 0.9% BD-rate reduction under low-delay P configuration.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Houqiang Li
4. Ming-Hsuan Yang
5. Trevor Darrell
6. Jitendra Malik
7. Yi Yang
8. Xiaogang Wang

In Signal Processing

1. Shanshe Wang
2. Shiqi Wang
3. Siwei Ma
4. Xinfeng Zhang
5. Zhenghui Zhao
6. Houqiang Li
7. Shuai Huo
8. Dong Liu

Citations in the corpus

• Learned Video Compression (2018, background): “These include improved motion compensation and interpolation [54, 21, 58, 31], intra-prediction coding [40], post-processing refinement [7, 55, 26, 56, 48, 57, 41, 17], and rate control [28].”
• Convolutional Neural Network-Based Motion Compensation Refinement for Video Coding (2018, methodology): “[7] proposed a CNN-based method for half-pixel interpolation in HEVC inter coding, where a trained CNN is adopted to replace the previous fixed-weight interpolation filters so as to generate motion compensated prediction signal when the MV is of half pixels.”
• CNN-Based Bi-Directional Motion Compensation for High Efficiency Video Coding (2018, methodology): “[13] introduced a CNN-based interpolation filter into the codec for improving the accuracy of the”