“Double-DIP”: Unsupervised Image Decomposition via Coupled Deep-Image-Priors

By Yossi Gandelsman, Assaf Shocher, Michal Irani. Computer Vision and Pattern Recognition, 2018

367 citations recorded by the source corpus.

Many seemingly unrelated computer vision tasks can be viewed as a special case of image decomposition into separate layers. For example, image segmentation (separation into foreground and background layers); transparent layer separation (into reflection and transmission layers); Image dehazing (separation into a clear image and a haze map), and more. In this paper we propose a unified framework for unsupervised layer decomposition of a single image, based on coupled "Deep-image-Prior" (DIP) networks. It was shown [Ulyanov et al] that the structure of a single DIP generator network is sufficient to capture the low-level statistics of a single image. We show that coupling multiple such DIPs provides a powerful tool for decomposing images into their basic components, for a wide variety of applications. This capability stems from the fact that the internal statistics of a mixture of layers is more complex than the statistics of each of its individual components. We show the power of this approach for Image-Dehazing, Fg/Bg Segmentation, Watermark-Removal, Transparency Separation in images and video, and more. These capabilities are achieved in a totally unsupervised way, with no training examples other than the input image/video itself.

View Paper: https://www.semanticscholar.org/paper/b32ada2aab1e76ba3d0125c7a15615a531c6ce88

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Neural Architecture Search for Deep Image Prior (2020, background): “Very recently, single image GAN [32] has been proposed to learn a multi-resolution model of appearance and DIP has been applied to image layer decomposition [13].”
CLEARER: Multi-Scale Neural Architecture Search for Image Restoration (2020, background): “In other words, these so-called learning-based methods substitute the explicit and handcrafted image priors with implicit and learning-based priors which are captured by neural architectures [41, 11, 29].”
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search (2020, methodology): “We present the visual comparisons with DoubleDIP [22] on the O-HAZE dataset [4].”
Neural Blind Deconvolution Using Deep Priors (2019, methodology): “As a result, we empirically ﬁnd that Double-DIP [6] performs poorly for blind deconvolution (see the results in Sec.”
NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction (2021, methodology): “[25] used multiple deep image prior networks for image decomposition in various applications.”
You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network (2020, methodology): “To enjoy the end-to-end optimization using the standard stochastic gradient methods, the reparameterization trick could be used to yield a lower bound estimator (Kingma and Welling, 2014).”