Multi-frame Quality Enhancement for Compressed Video
By Ren Yang, Mai Xu, Zulin Wang, Tianyi Li. 2018
235 citations recorded by the source corpus.
The past few years have witnessed great success in applying deep learning to enhance the quality of compressed image/video. The existing approaches mainly focus on enhancing the quality of a single frame, ignoring the similarity between consecutive frames. In this paper, we investigate that heavy quality fluctuation exists across compressed video frames, and thus low quality frames can be enhanced using the neighboring high quality frames, seen as Multi-Frame Quality Enhancement (MFQE). Accordingly, this paper proposes an MFQE approach for compressed video, as a first attempt in this direction. In our approach, we firstly develop a Support Vector Machine (SVM) based detector to locate Peak Quality Frames (PQFs) in compressed video. Then, a novel Multi-Frame Convolutional Neural Network (MF-CNN) is designed to enhance the quality of compressed video, in which the non-PQF and its nearest two PQFs are as the input. The MF-CNN compensates motion between the non-PQF and PQFs through the Motion Compensation subnet (MC-subnet). Subsequently, the Quality Enhancement subnet (QE-subnet) reduces compression artifacts of the non-PQF with the help of its nearest PQFs. Finally, the experiments validate the effectiveness and generality of our MFQE approach in advancing the state-of-the-art quality enhancement of compressed video. The code of our MFQE approach is available at https://github.com/ryangBUAA/MFQE.git.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Xiaopeng Hong
- Dacheng Tao
- Guoying Zhao
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Li Fei-Fei
Citations in the corpus
- Learned Video Compression (2018, background): “These include improved motion compensation and interpolation [49, 16, 53, 26], intra-prediction coding [35], post-processing refinement [7, 50, 21, 51, 43, 52, 36, 14], and rate control [23].”
- Remote Heart Rate Measurement From Highly Compressed Facial Videos: An End-to-End Deep Learning Solution With Video Enhancement (2019, background): “Fueled by the high performance of deep learning, several works introduce it to enhance the quality of compressed videos and get promising results, including ARCNN [5], deep residual denoising neural networks (DnCNN) [39], generative adversarial networks [6] and multi-frame quality enhancement networ”
- Image and Video Compression With Neural Networks: A Review (2019, unknown): “quality frames in compressed video [110].”