Real-Time High-Resolution Background Matting
By Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian Curless, S. M. Seitz, Ira Kemelmacher-Shlizerman. Computer Vision and Pattern Recognition, 2020
306 citations recorded by the source corpus.
We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the foreground layer. The main challenge is to compute a high-quality alpha matte, preserving strand-level hair details, while processing high-resolution images in real-time. To achieve this goal, we employ two neural networks; a base network computes a low-resolution result which is refined by a second network operating at high-resolution on selective patches. We introduce two large-scale video and image matting datasets: VideoMatte240K and PhotoMatte13K/85. Our approach yields higher quality results compared to the previous state-of-the-art in background matting, while simultaneously yielding a dramatic boost in both speed and resolution.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/1f476738e6d2950ab2ca08ac852764640c8e9d93
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Yebin Liu
- Zerong Zheng
- Liam Schoneveld
- Ruizhi Shao
- Simon Giebenhain
- Boning Liu
- Hanzhang Tu
- Matthias Nießner
Citations in the corpus
- GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (2023, background): “The set of Gaussian splats can deviate from the mesh surface to compensate the absence or inaccuracy of the morphable model, exhibiting astonishing ability to model fine details on human heads.”
- Tensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering (2022, methodology): “Since we focus on the dynamic objects in the scene, we adopt BackGroundMattingV2 [3] to segment the foreground objects in the images for our dataset.”