CRF Learning with CNN Features for Image Segmentation

By Fayao Liu, Guosheng Lin, Chunhua Shen. 2015

217 citations recorded by the source corpus.

Conditional Random Rields (CRF) have been widely applied in image segmentations. While most studies rely on hand-crafted features, we here propose to exploit a pre-trained large convolutional neural network (CNN) to generate deep features for CRF learning. The deep CNN is trained on the ImageNet dataset and transferred to image segmentations here for constructing potentials of superpixels. Then the CRF parameters are learnt using a structured support vector machine (SSVM). To fully exploit context information in inference, we construct spatially related co-occurrence pairwise potentials and incorporate them into the energy function. This prefers labelling of object pairs that frequently co-occur in a certain spatial layout and at the same time avoids implausible labellings during the inference. Extensive experiments on binary and multi-class segmentation benchmarks demonstrate the promise of the proposed method. We thus provide new baselines for the segmentation performance on the Weizmann horse, Graz-02, MSRC-21, Stanford Background and PASCAL VOC 2011 datasets. HighlightsA deep CNN pretrained on ImageNet generalizes well to various segmentation datasets.Deep features significantly outperform BoW and unsupervisd feature learning.Combining deep CNN features with CRF yields new state-of-the-art results.Incorporating spatial related co-occurrence potentials further improves the accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Taskonomy: Disentangling Task Transfer Learning (2018, methodology): “ormation needed for mapping the input to the output. These representations, however, can transmit statistics useful for solving other outputs (tasks), presumably if the tasks are related in some form [83,19,58,46]. This is the basis of our approach: we computes an afﬁnity matrix among tasks based on”
Spatial, Structural and Temporal Feature Learning for Human Interaction Prediction (2016, methodology): “As CNN (Convolutional Neural Network) features have been shown transferable across domains [6], and have been successfully applied in a variety of visual recognition tasks such as object detection, image classification and segmentation [7, 8, 9], this paper applies a CNN model, pre-trained with Imag”
ReSeg: A Recurrent Neural Network for Object Segmentation (2015, background): “, 2015) and (Liu et al., 2015), and we think that scaling to larger datasets should allow for the strengths of this model to become more apparent.”
Wound intensity correction and segmentation with convolutional neural networks (2017, unknown): “Liu et al. [23] used CNNs to generate deep features for learning conditional random ﬁelds.”
Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction (2016, background): “CNN pre-trained models have been shown to be transferable across domains [29], and have achieved a better performance than hand-crafted features in a variety of visual recognition tasks [30], [31], [32], [33], [34].”
Multi-scale context for scene labeling via flexible segmentation graph (2016, background): “Recently, an alternative strategies to implicitly capture contextual clues appeared in [12,14,21] and [47] using deep learning techniques.”
Scene parsing using inference Embedded Deep Networks (2016, methodology): “[45] combine the CNN and CRF to complete the image segmentation task.”