Reducing Complexity of HEVC: A Deep Learning Approach

By Mai Xu, Tianyi Li, Zulin Wang, Xin Deng, Ren Yang, Zhenyu Guan. IEEE Transactions on Image Processing, 2017

303 citations recorded by the source corpus.

High efficiency video coding (HEVC) significantly reduces bit rates over the preceding H.264 standard but at the expense of extremely high encoding complexity. In HEVC, the quad-tree partition of the coding unit (CU) consumes a large proportion of the HEVC encoding complexity, due to the brute-force search for rate-distortion optimization (RDO). Therefore, this paper proposes a deep learning approach to predict the CU partition for reducing the HEVC complexity at both intra-and inter-modes, which is based on convolutional neural network (CNN) and long- and short-term memory (LSTM) network. First, we establish a large-scale database including substantial CU partition data for the HEVC intra- and inter-modes. This enables deep learning on the CU partition. Second, we represent the CU partition of an entire coding tree unit in the form of a hierarchical CU partition map (HCPM). Then, we propose an early terminated hierarchical CNN (ETH-CNN) for learning to predict the HCPM. Consequently, the encoding complexity of intra-mode HEVC can be drastically reduced by replacing the brute-force search with ETH-CNN to decide the CU partition. Third, an ETH-LSTM is proposed to learn the temporal correlation of the CU partition. Then, we combine the ETH-LSTM and the ETH-CNN to predict the CU partition for reducing the HEVC complexity at inter-mode. Finally, experimental results show that our approach outperforms the other state-of-the-art approaches in reducing the HEVC complexity at both intra- and inter-modes.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/94ce9fb2e490f580f50f0ce73d57805ed92d5bc6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Mai Xu
2. Zulin Wang
3. Ren Yang
4. Zhenyu Guan
5. Shanshe Wang
6. Shiqi Wang
7. Tie Liu
8. Siwei Ma

In Signal Processing

1. Mai Xu
2. Zulin Wang
3. Ren Yang
4. Shanshe Wang
5. Shiqi Wang
6. Siwei Ma
7. Xinfeng Zhang
8. Chuanmin Jia

Citations in the corpus

• Enhancing Quality for HEVC Compressed Videos (2017, background): “[28] proposed reducing HEVC encoding complexity by deep networks.”
• Image and Video Compression With Neural Networks: A Review (2019, methodology): “predicted the entire CTU partition structure by using both CNN and LSTM to determine whether the mode decision should be early terminated [120].”