High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks

By Haohan Wang, Xindi Wu, Pengcheng Yin, Eric P. Xing. 2019

708 citations recorded by the source corpus.

We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN). We first notice CNN's ability in capturing the high-frequency components of images. These high-frequency components are almost imperceptible to a human. Thus the observation leads to multiple hypotheses that are related to the generalization behaviors of CNN, including a potential explanation for adversarial examples, a discussion of CNN's trade-off between robustness and accuracy, and some evidence in understanding training heuristics.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Scaling & Learning Theory

Citations in the corpus

Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “Recent work has connected lack of robustness to computer vision models’ encoding counterintuitive features (Ilyas et al., 2019; Geirhos et al., 2019; Yin et al., 2019; Wang et al., 2020).”
Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, background): “Other theories focus on the Fourier perspective of the robustness [67, 68], showing that adversarial training might be preventing the network from learning the high frequency signals of the input image.”
FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, background): “Note several prior works exploited different FC properties successfully in other tasks such as data compression [74] and supervised learning [73].”
Generalized Out-of-Distribution Detection: A Survey (2021, methodology): “Methods such as CNN kernel smoothing [102] and”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “In fact, frequency analysis is a common and important way in digital image processing and has been widely applied to various tasks in computer vision [44, 47, 41, 18].”
Towards Frequency-Based Explanation for Robust CNN (2020, background): “By observing adversarial defended models, Wang et al. (2019) concludes that smoothing the CNN kernels helps to enforce the model to use features of low frequencies.”
Self-Challenging Improves Cross-Domain Generalization (2020, background): “Nowadays, deep neural networks have exhibited remarkable empirical results over various computer vision tasks, yet these impressive performances seem unmet when the models are tested with the samples in irregular qualities [32] (i.”
Wavelet Convolutions for Large Receptive Fields (2024, background): “This finding was somewhat surprising because convolutional layers usually tend to respond to high-frequencies in the input [17,19,54,59].”
Inception Transformer (2022, background): “Unlike ViTs, they cover more local information through local convolution within the receptive fields, thus effectively extracting high-frequency representations [19, 20].”
ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (2021, background): “Image semantics contained in CNN features [38], segmentation masks [31], and low-resolution images [26, 46] are actively used as conditions in generative models.”
Focal Frequency Loss for Image Reconstruction and Synthesis (2020, background): “The application areas of the frequency domain analysis have been further extended, including media forensics [59, 72, 22, 12], super-resolution [13, 63], generalization analysis [58, 20], magnetic resonance imaging [53], image rescaling [64], etc.”