Learning in the Frequency Domain
By Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-kuang Chen, Fengbo Ren. 2020
607 citations recorded by the source corpus.
Deep neural networks have achieved remarkable success in computer vision tasks. Existing neural networks mainly operate in the spatial domain with fixed input sizes. For practical applications, images are usually large and have to be downsampled to the predetermined input size of neural networks. Even though the downsampling operations reduce computation and the required communication bandwidth, it removes both redundant and salient information obliviously, which results in accuracy degradation. Inspired by digital signal processing theories, we analyze the spectral bias from the frequency perspective and propose a learning-based frequency selection method to identify the trivial frequency components which can be removed without accuracy loss. The proposed method of learning in the frequency domain leverages identical structures of the well-known neural networks, such as ResNet-50, MobileNetV2, and Mask R-CNN, while accepting the frequency-domain information as the input. Experiment results show that learning in the frequency domain with static channel selection can achieve higher accuracy than the conventional spatial downsampling approach and meanwhile further reduce the input data size. Specifically for ImageNet classification with the same input size, the proposed method achieves 1.60% and 0.63% top-1 accuracy improvements on ResNet-50 and MobileNetV2, respectively. Even with half input size, the proposed method still improves the top-1 accuracy on ResNet-50 by 1.42%. In addition, we observe a 0.8% average precision improvement on Mask R-CNN for instance segmentation on the COCO dataset.
Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
In Electrical and Electronic Engineering
Citations in the corpus
- MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, background): “Some of these frequency-aware methods aim to reduce the computing cost with frequency transformation [83], [84], thus improving the network effi-ciency.”
- FcaNet: Frequency Channel Attention Networks (2020, background): “Moreover, some methods [21, 41] have shown CNNs prefer low-frequency information.”
- Fast Vision Transformers with HiLo Attention (2022, background): “Based on this insight, a plethora of solutions have been proposed for image superresolution [53, 14], generalization [19], image re-scaling [44] and neural network compression [47, 5].”
- FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, background): “Note several prior works exploited different FC properties successfully in other tasks such as data compression [74] and supervised learning [73].”
- QARV: Quantization-Aware ResNet VAE for Lossy Image Compression (2023, background): “However, applying recent approaches for compressed domain object recognition [74], [75], [76] to a hierarchical architecture is non-trivial, and we leave it to future work.”
- Focal Frequency Loss for Image Reconstruction and Synthesis (2020, background): “Besides, several studies have incorporated frequency analysis with network compression [16, 66, 68, 6, 17] and feature reduction [35, 61] to accelerate the training and inference of networks.”
- Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection (2021, background): “In order to tackle this problem, inspired by [16, 44, 31, 26], we pro-pose an adaptive frequency feature generation module (AF-FGM) to efficiently mine subtle artifacts from the frequency domain.”