Loss Max-Pooling for Semantic Image Segmentation

By Samuel Rota Bulò, Gerhard Neuhold, Peter Kontschieder. 2017

126 citations recorded by the source corpus.

We introduce a novel loss max-pooling concept for handling imbalanced training data distributions, applicable as alternative loss layer in the context of deep neural networks for semantic image segmentation. Most real-world semantic segmentation datasets exhibit long tail distributions with few object categories comprising the majority of data and consequently biasing the classifiers towards them. Our method adaptively re-weights the contributions of each pixel based on their observed losses, targeting under-performing classification results as often encountered for under-represented object classes. Our approach goes beyond conventional cost-sensitive learning attempts through adaptive considerations that allow us to indirectly address both, inter- and intra-class imbalances. We provide a theoretical justification of our approach, complementary to experimental analyses on benchmark datasets. In our experiments on the Cityscapes and Pascal VOC 2012 segmentation datasets we find consistently improved results, demonstrating the efficacy of our approach.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Focal Loss for Dense Object Detection (2017, background): “A common solution is to perform some form of hard negative mining [32, 36, 8, 30, 21] that samples hard examples during training or more complex sampling/reweighing schemes [2].”
Strip Pooling: Rethinking Spatial Pooling for Scene Parsing (2020, methodology): “For continuous feature space learning, prior work use multi-scale feature aggregation [37, 5, 33, 18, 42, 31, 32, 2, 44, 4, 48, 17] to fuse the contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple fields-of-view.”
Remix: Rebalanced Mixup (2020, background): “Re-Weighting Re-weighting (cost-sensitive learning) focuses on tuning cost or loss to redefine the importance of each class or sample [6,34,15,3].”
Focal Loss for Dense Object Detection (2017, background): “A common solution is to perform some form of hard negative mining [17], [19], [20], [16], [9] that samples hard examples during training or more complex sampling/reweighing schemes [34].”
Online Multi-Object Tracking with Dual Matching Attention Networks (2018, methodology): “Recently, several methods [6,34] re-weight the contribution of each sample based on the observed loss and demonstrate signiﬁcant improvements on segmentation and detection tasks.”
Seamless Scene Segmentation (2019, methodology): “Other works like [45, 53] are addressing the problem of class sample imbalance by introducing loss-guided, pixel-wise gradient reweighting schemes.”
Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection (2019, methodology): “To address this problem, a common solution is to perform some forms of hard mining [44] [45] that samples hard samples during training or more complex sampling/reweighting schemes [46].”
VITAL: VIsual Tracking via Adversarial Learning (2018, background): “Hard negative mining [15, 46] and reweighing training data [45, 35] are useful to alleviate the class imbalance problem to some extent.”
EfficientPS: Efficient Panoptic Segmentation (2020, methodology): “We use the weighted per-pixel log-loss (Bulo et al, 2017) for training which is given by Lpp(Θ) =−∑ i j wi j log pi j(p∗i j), (1) where p∗i, j is the groundtruth for a given image, pi, j is the predicted probability for the pixel (i, j) being assigned class c ∈ p, wi j = 4WH if pixel (i, j) belongs…”
Deep Imbalanced Learning for Face Recognition and Attribute Prediction (2018, methodology): “use from the chosen class, we borrow the idea of [23] to pick I 1 as the one with the highest observed loss in class.”
Deep learning based brain tumor segmentation: a survey (2020, background): “The imbalanced issue affects the data-driven learning algorithm as the extracted features may be highly influenced by large tumor regions [28].”