SegFix: Model-Agnostic Boundary Refinement for Segmentation

By Yuhui Yuan, Jingyi Xie, Xilin Chen, Jingdong Wang. 2020

238 citations recorded by the source corpus.

We present a model-agnostic post-processing scheme to improve the boundary quality for the segmentation result that is generated by any existing segmentation model. Motivated by the empirical observation that the label predictions of interior pixels are more reliable, we propose to replace the originally unreliable predictions of boundary pixels by the predictions of interior pixels. Our approach processes only the input image through two steps: (i) localize the boundary pixels and (ii) identify the corresponding interior pixel for each boundary pixel. We build the correspondence by learning a direction away from the boundary pixel to an interior pixel. Our method requires no prior information of the segmentation models and achieves nearly real-time speed. We empirically verify that our SegFix consistently reduces the boundary errors for segmentation results generated from various state-of-the-art models on Cityscapes, ADE20K and GTA5. Code is available at: this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “These methods can either be a part of the network architecture [28, 29] or serve as a postprocessing step [30, 31].”
Scaling Wide Residual Networks for Panoptic Segmentation (2020, unknown): “7 HRNetV2 + OCR + SegFix [99, 100, 79] C, MV - - 84.”
Segment Anything in High Quality (2023, background): “High-quality Segmentation Existing works for high-quality segmentation are mostly trained for a specific segmentation task, like image and video instance segmentation [22, 19, 20, 39, 43], semantic segmentation [30, 52, 38, 49] or panoptic segmentation [9], in a close-world paradigm.”
SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation (2022, background): “There are different types of decoders for different goals, including achieving multi-scale receptive fields [94, 7, 78], collecting multi-scale semantics [64, 80, 8], enlarging receptive field [5, 4, 62], strengthening edge features [95, 2, 16, 42, 90], and capturing global context [19, 34, 89, 40, ”
Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision (2021, methodology): “Improving the segmentation quality on the edge areas include Gated-SCNN [31], PointRend [20], and SegFix [39].”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “Since it was proposed, numerous efforts have been devoted to improving FCN, by, for example, enlarging the receptive field [15, 16, 25, 124, 128, 135]; strengthening context cues [4, 43, 47, 48, 56, 57, 70, 75, 77, 81, 90, 126, 128, 129, 132, 138, 141]; leveraging boundary information [6, 14, 27, 66”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, unknown): “This concept was further improved in different aspects including increasing the receptive field while preserving spatial details [2, 3, 6, 48, 65, 74, 83], integrating context information [30,75,76,78,91], attention mechanisms [18,32,69,84], refining boundaries [15,36,77], and Transformer-based arch”