Interpreting Super-Resolution Networks with Local Attribution Maps

By Jinjin Gu, Chao Dong. Computer Vision and Pattern Recognition, 2020

353 citations recorded by the source corpus.

Image super-resolution (SR) techniques have been developing rapidly, benefiting from the invention of deep networks and its successive breakthroughs. However, it is acknowledged that deep learning and deep neural networks are difficult to interpret. SR networks inherit this mysterious nature and little works make attempt to understand them. In this paper, we perform attribution analysis of SR networks, which aims at finding the input pixels that strongly influence the SR results. We propose a novel attribution approach called local attribution map (LAM), which inherits the integral gradient method yet with two unique features. One is to use the blurred image as the baseline input, and the other is to adopt the progressive blurring function as the path function. Based on LAM, we show that: (1) SR networks with a wider range of involved input pixels could achieve better performance. (2) Attention networks and non-local networks extract features from a wider range of input pixels. (3) Comparing with the range that actually contributes, the receptive field is large enough for most deep networks. (4) For SR networks, textures with regular stripes or grids are more likely to be noticed, while complex semantics are difficult to utilize. Our work opens new directions for designing SR networks and interpreting low-level vision deep models.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8e5f8c807d38a6bd603ef86902ef57b35437d9d0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Radu Timofte
3. Chen Change Loy
4. Chao Dong
5. Yu Qiao
6. Lei Zhang
7. Ming-Hsuan Yang
8. Dacheng Tao

Citations in the corpus

• MambaIR: A Simple Baseline for Image Restoration with State-Space Model (2024, result): “…resulted structure will be similar to Transformer, also leads to unfavorable results, indicating that although both SSMs and Attention have the global modeling ability, the behavior of these two modules is different and thus accustomed block structure should be considered for further improvements.”
• Dual Aggregation Transformer for Image Super-Resolution (2023, methodology): “Firstly , we apply the LAM [7] to visualize the range of information utilization.”
• Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, background): “Existing frameworks often overlook or implicitly handle this understanding [24, 29].”
• Efficient and Explicit Modelling of Image Hierarchies for Image Restoration (2023, background): “On the other hand, it is shown in recent works [7,25] that even a slight increase in window size can lead to better image restoration.”