Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal

By Jifeng Wang, Xiang-Yang Li, Le Hui, Jian Yang. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

517 citations recorded by the source corpus.

Understanding shadows from a single image consists of two types of task in previous studies, containing shadow detection and shadow removal. In this paper, we present a multi-task perspective, which is not embraced by any existing work, to jointly learn both detection and removal in an end-to-end fashion that aims at enjoying the mutually improved benefits from each other. Our framework is based on a novel STacked Conditional Generative Adversarial Network (ST-CGAN), which is composed of two stacked CGANs, each with a generator and a discriminator. Specifically, a shadow image is fed into the first generator which produces a shadow detection mask. That shadow image, concatenated with its predicted mask, goes through the second generator in order to recover its shadow-free image consequently. In addition, the two corresponding discriminators are very likely to model higher level relationships and global scene characteristics for the detected shadow region and reconstruction via removing shadows, respectively. More importantly, for multi-task learning, our design of stacked paradigm provides a novel view which is notably different from the commonly used one as the multi-branch version. To fully evaluate the performance of our proposed framework, we construct the first large-scale benchmark with 1870 image triplets (shadow image, shadow mask image, and shadow-free image) under 135 scenes. Extensive experimental results consistently show the advantages of STC-GAN over several representative state-of-the-art methods on two large-scale publicly available datasets and our newly released one.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7ae613ab4925975be47c6d5418a5153b73b4ba55

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Vishal M. Patel
2. He Zhang
3. P. Heng
4. Liang Wan
5. Lanyun Zhu
6. Zhihao Chen
7. Lei Zhu
8. Tianrun Chen

Citations in the corpus

• Densely Connected Pyramid Dehazing Network (2018, background): “The success of GANs in synthesizing realistic images has led researchers to explore the adversarial loss for various low-level vision applications such as text-to-image synthesis[32, 52, 54, 6], image-image translation [18, 28, 46] and other applications [23, 50, 55, 58, 45, 38, 44, 31].”
• SAM-Adapter: Adapting Segment Anything in Underperformed Scenes (2023, methodology): “For shadow detection, we use ISTD dataset [47], which contains 1,330 training images and 540 test images.”
• A Multi-Task Mean Teacher for Semi-Supervised Shadow Detection (2020, methodology): “SBU [33] UCF [42] ISTD [35] Method Year BER ↓ Shadow ↓ Non Shad.”