Loss Functions for Neural Networks for Image Processing

By Hang Zhao, Orazio Gallo, Iuri Frosio, Jan Kautz. arXiv.org, 2015

314 citations recorded by the source corpus.

Neural networks are becoming central in several areas of computer vision and image processing and different architectures have been proposed to solve specific problems. The impact of the loss layer of neural networks, however, has not received much attention in the context of image processing: the default and virtually only choice is L2. In this paper, we bring attention to alternative choices for image restoration. In particular, we show the importance of perceptually-motivated losses when the resulting image is to be evaluated by a human observer. We compare the performance of several losses, and propose a novel, differentiable error function. We show that the quality of the results improves significantly with better loss functions, even when the network architecture is left unchanged.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/401ec2881b20ce7d6dc07c274a1d8f821d4c3841

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Data-driven synthesis of smoke flows with CNN-based feature descriptors (2017, unknown): “Furthermore, previous work [Zhao et al. 2015] has shown that an additional L 1 loss term with a small weight can be added to the generator to ensure that its output stays close to the ground truth y .”
Multiresolution Knowledge Distillation for Anomaly Detection (2020, methodology): “SSIM-AE [10] trains an AE with SSIM loss [54] instead of MSE causing to perform just better on defect segmentation.”
Deep Fluids: A Generative Network for Parameterized Fluid Simulations (2018, background): “While previous works have proposed loss functions for natural images, e.g., L p norms, MS-SSIM [ZGFK17], and perceptual losses [JAFF16, LTH ∗ 17], accurate reconstructions of velocity ﬁelds have not been investigated.”
Deep Learning for Single Image Super-Resolution: A Brief Review (2018, methodology): “As reported in [112], when MAE is used to optimize an NN, the NN tends to converge faster and produce better results.”
GLADNet: Low-Light Enhancement Network with Global Awareness (2018, background): “L2 norm to better remove noise and ringing artifacts in the enhanced results [10].”
Enhanced Deep Residual Networks for Single Image Super-Resolution (2017, background): “[35] reported that training with L2 loss does not guarantee better performance compared to other loss functions in terms of PSNR and SSIM.”
Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints (2018, methodology): “Similar to [11, 31], we use it as a loss term in the training process.”
Multiscale brain MRI super-resolution using deep 3D convolutional networks (2019, background): “Moreover, whereas the classical MSE-based loss attempts to recover the smooth component, perceptual losses (Ledig et al. 2017; Johnson et al. 2016; Zhao et al. 2017) are proposed for natural image SR to better reconstruct ﬁne details and edges.”
Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer (2017, methodology): “To preserve the content and structure of the input image, SSIM loss [33] is used.”
Single Image Reflection Separation with Perceptual Losses (2018, methodology): “When we replace the feature loss Lfeat with a L 1 loss in color space, the output images tend to be overly-smooth; similar observation is also discussed in [34, 12].”
An Introduction to Image Synthesis with Generative Adversarial Nets (2018, background): “The reason for using L1 instead of L2 is that L1 produces less blurry images [70].”
Scale Aggregation Network for Accurate and Efficient Crowd Counting (2018, methodology): “Inspired by [10], we use a combination of Euclidean loss and local pattern consistency loss to exploit the local correlation in density maps.”
DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs (2017, background): “While 2 loss function is better suited for classification tasks, they may not be a correct choice for image processing tasks [29].”