DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
By Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin P. Murphy, Alan L. Yuille. 2016
21,279 citations recorded by the source corpus.
In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or ‘atrous convolution’, as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed “DeepLab” system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7 percent mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Jitendra Malik
- Andreas Geiger
- Luc Van Gool
- Marc Pollefeys
- Silvio Savarese
- Kristen Grauman
- Vladlen Koltun
- Abhinav Kumar Gupta
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Yi Yang
- Alan L. Yuille
Citations in the corpus
- Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “5 Besides, MoCo is on par on Cityscapes instance segmentation, and lags behind on VOC semantic segmentation; we show another comparable case on iNatu-ralist [57] in appendix.”
- Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers (2020, background): “Instead, memory storage/movement is the limiting factor in image classi-fication (Gomez et al., 2017), semantic segmentation (Chen et al., 2017), language modeling (Kitaev et al., 2020), and other tasks (Jain et al., 2020).”
- Do Better ImageNet Models Transfer Better? (2018, methodology): “A parallel story likewise appears in image segmentation models [7], although it has not been as systematically explored.”
- Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “Secondly, we do not adopt the atrous spatial pyramid pooling module (ASPP) [14,15], since our axial-attention block could also efficiently encode the multiscale or global information.”
- Local Relation Networks for Image Recognition (2019, methodology): “This has been done to enlarge the receptive field, such as through atrous/dilated convolution [4, 29], and to enhance the ability to model geometric deformation, via active [14] and deformable convolution [6, 32].”
- Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, methodology): “These visual representations have been shown to be effective on a wide range of computer vision tasks [1, 4, 14, 22, 29, 33, 36].”
- Making Convolutional Networks Shift-Invariant Again (2019, methodology): “Deep networks can also be evaluated densely, by removing striding and making appropriate changes to subsequent layers by using ´a trous /dilated convolutions (Chen et al., 2015; 2018; Yu & Koltun, 2016; Yu et al., 2017).”
- Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “Future work will study how this method applies to other machine learning domains, where data augmentation is known to improve predictive performance, such as image segmentation [3], 3-”
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, background): “tection [29, 23], semantic segmentation [1, 24], and video analysis [27, 31].”
- Unified Perceptual Parsing for Scene Understanding (2018, methodology): “Same as [36], we use the “poly” learning rate policy where the learning rate at current iteration equals the initial learning rate multiplying 1 − iter max iter power .”
- Semantic Understanding of Scenes Through the ADE20K Dataset (2016, methodology): “We integrate the proposed module on two baseline networks Segnet [1] and DilatedNet [4, 25].”
- Colorful Image Colorization (2016, methodology): “use a two-stream architecture in which they fuse global and local features, and we use a single-stream, VGG-styled network with added depth and dilated convolutions [26,27].”
- UPSNet: A Unified Panoptic Segmentation Network (2019, methodology): “For PSPNet, we use ‘poly’ learning rate schedule as in [3] and train 220K, 18K and 76K on COCO, Cityscapes and our dataset with minibatch size 16.”
- Panoptic Segmentation (2018, background): “Stuff classifiers are usually built on fully convolutional nets [30] with dilations [51, 5] while object detectors often use object proposals [15] and are region-based [37, 14].”