A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding

By Yuanying Dai, Dong Liu, Feng Wu. Conference on Multimedia Modeling, 2016

314 citations recorded by the source corpus.

Lossy image and video compression algorithms yield visually annoying artifacts including blocking, blurring, and ringing, especially at low bit-rates. To reduce these artifacts, post-processing techniques have been extensively studied. Recently, inspired by the great success of convolutional neural network (CNN) in computer vision, some researches were performed on adopting CNN in post-processing, mostly for JPEG compressed images. In this paper, we present a CNN-based post-processing algorithm for High Efficiency Video Coding (HEVC), the state-of-the-art video coding standard. We redesign a Variable-filter-size Residue-learning CNN (VRCNN) to improve the performance and to accelerate network training. Experimental results show that using our VRCNN as post-processing leads to on average 4.6% bit-rate reduction compared to HEVC baseline. The VRCNN outperforms previously studied networks in achieving higher bit-rate reduction, lower memory cost, and multiplied computational speedup.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/36d2d7fb483960141c0c55ae781a12bc07b467f2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Mai Xu
2. Ren Yang
3. Zulin Wang
4. Siwei Ma
5. Shiqi Wang
6. Shanshe Wang
7. Xinfeng Zhang
8. Zhenghui Zhao

In Signal Processing

1. Mai Xu
2. Zulin Wang
3. Ren Yang
4. Shanshe Wang
5. Shiqi Wang
6. Siwei Ma
7. Xinfeng Zhang
8. Zhenghui Zhao

Citations in the corpus

• A convolutional neural network approach for half-pel interpolation in video coding (2017, background): “[12] propose to learn a CNN for post-processing in video coding, and demonstrate on average 4.”
• Decoder-side HEVC quality enhancement with scalable convolutional neural network (2017, methodology): “EXPERIMENTS In this section, the experimental results are presented to validate the effectiveness of our DS-CNN approach, comparing with the latest quality enhancement approaches AR-CNN [14] and VRCNN [18].”
• Multi-frame Quality Enhancement for Compressed Video (2018, methodology): “For the quality enhancement of compressed video, the Variable-filter-size Residue-learning CNN (VRCNN) [8] was proposed to replace the inloop filters for HEVC intra-coding.”
• CNN based post-processing to improve HEVC (2017, methodology): “[2] propose a Variable-filter-size Residue-learning CNN (VRCNN) to replace both deblocking filter and SAO.”
• A Practical Convolutional Neural Network as Loop Filter for Intra Frame (2018, methodology): “[3] adopts a CNN with residual learning structure with variable filter size for acceleration .”
• Convolutional Neural Network-Based Motion Compensation Refinement for Video Coding (2018, methodology): “[6] proposed a CNN-based method for post-processing in HEVC intra coding, where a trained CNN, instead of hand-crafted deblocking or other post-processing filters, is adopted to refine the reconstructed image; Yan et al.”
• Enhancing HEVC Compressed Videos with a Partition-Masked Convolutional Neural Network (2018, methodology): “Five methods are compared in Table 2: (1) VRCNN [4] which is a benchmark CNN-based compressed-video enhancement method; (2) QECNN-P [5] which is a state-ofthe-art compressed-video enhancement method for P frames in HEVC; (3) Our (1-in), which is the single input baseline of our approach; (4) VRCNN+M”
• Enhancing Quality for HEVC Compressed Videos (2017, methodology): “Then, our QE-CNN method and the AR-CNN, VRCNN and DCAD methods are applied to enhance the quality of the encoded HEVC sequences.”
• Image and Video Compression With Neural Networks: A Review (2019, methodology): “The CNNMCR jointly employs the motion compensated prediction and its neighboring reconstructed blocks as input of VRCNN, which is trained by minimizing the mean square errors between the input and its corresponding original signal.”