Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology

By David Tellez, Geert J. S. Litjens, Péter Bándi, Wouter Bulten, John-Melle Bokhorst, Francesco Ciompi, J. A. van der Laak. Medical Image Anal., 2019

642 citations recorded by the source corpus.

Stain variation is a phenomenon observed when distinct pathology laboratories stain tissue slides that exhibit similar but not identical color appearance. Due to this color shift between laboratories, convolutional neural networks (CNNs) trained with images from one lab often underperform on unseen images from the other lab. Several techniques have been proposed to reduce the generalization error, mainly grouped into two categories: stain color augmentation and stain color normalization. The former simulates a wide variety of realistic stain variations during training, producing stain-invariant CNNs. The latter aims to match training and test color distributions in order to reduce stain variation. For the first time, we compared some of these techniques and quantified their effect on CNN classification performance using a heterogeneous dataset of hematoxylin and eosin histopathology images from 4 organs and 9 pathology laboratories. Additionally, we propose a novel unsupervised method to perform stain color normalization using a neural network. Based on our experimental results, we provide practical guidelines on how to use stain color augmentation and stain color normalization in future computational pathology applications.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/d6e975989345b69f539dbb8f22cb3437f5cc5039

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Deep learning in histopathology: the path to the clinic (2021, methodology): “…data augmentation techniques, with a particular focus on color augmentation, to mimic differences in staining from different pathology laboratories 120 : image patches can be transformed before being used in the training process by applying random rotation, flipping, addition of noise, blurring…”
A Multi-Organ Nucleus Segmentation Challenge (2020, background): “While based on a prior idea that color normalization can improve performance of segmentation tasks [1], [52], it is becoming apparent that color augmentation (jitter) trains more robust segmentation models [53].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “The Camelyon17-wilds patch-based dataset is similar to one of the datasets used in Tellez et al. (2019), which was also derived from the CAMELYON17 challenge; there, only one hospital is used as the training set, and the other hospitals are all part of the test set.”
Measuring Domain Shift for Deep Learning in Histopathology (2020, background): “Tellez et al. [37] investigate how much improvement color augmentation and/or stain normalization have on generalization performance to new datasets.”
Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis (2019, methodology): “…architecture with default training parameters and the modified algorithm, with additional data augmentation to supersede any pre-existing JPEG patterns, and heavy color augmentation to overcome systematic differences in hematoxylin and eosin staining (random hue rotations by –90 to 90°) 55 .”
The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis (2020, background): “Since histopathological images do not have a canonical orientation, most of the authors applied data augmentation to increase the robustness of their network [97,98].”
Artificial intelligence for digital and computational pathology (2023, background): “Only through the concerted effort of multi-institutional data collection initiatives, open-source software packages and continued technical inspiration from advancements in computer vision and AI research can the said goals be reached.”