End-to-End Instance Segmentation and Counting with Recurrent Attention

By Mengye Ren, Richard S. Zemel. 2016

69 citations recorded by the source corpus.

While convolutional neural networks have gained impressive success recently in solving structured prediction problems such as semantic segmentation, it remains a challenge to differentiate individual object instances in the scene. Instance segmentation is very important in a variety of applications, such as autonomous driving, image captioning, and visual question answering. Techniques that combine large graphical models with low-level vision have been proposed to address this problem; however, we propose an end-to-end recurrent neural network (RNN) architecture with an attention mechanism to model a human-like counting process, and produce detailed instance segmentations. The network is jointly trained to sequentially produce regions of interest as well as a dominant object segmentation within each region. The proposed model achieves state-of-the-art results on the CVPPP leaf segmentation dataset and KITTI vehicle segmentation dataset.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

InstanceCut: From Edges to Instances with MultiCut (2016, background): “In [43] an attention-based recurrent neural network is presented.”
Semantic Instance Segmentation with a Discriminative Loss Function (2017, methodology): “Two popular approaches introduce a multi-stage pipeline with object proposals [17, 9, 28, 29, 12, 13, 42, 4], or train a recurrent network end-to-end with a custom loss function that outputs instances sequentially [26, 31, 30].”
Deep fruit detection in orchards (2016, background): “which uniquely identify objects in a scene [22], may be suitable for fruit disambiguation in clusters.”
MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving (2016, methodology): “Recently, several methods have proposed to use a single deep network that is trainable end-to-end to directly perform detection [44, 38, 39, 27].”
Joint Graph Decomposition & Node Labeling: Problem, Algorithms, Applications (2016, background): “Instance-separating semantic segmentation [7, 8, 20, 27, 28, 29, 37, 38] can be seen as a task requiring two classes of decisions: To every point in an image, one needs to assign a label that identifies a class of objects (e.”
Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images (2018, unknown): “…are applied to a variety of problems, including image classification (Jetley et al., 2018; Wang et al., 2017a; Zhao et al., 2017), segmentation (Ren and Zemel, 2016), action recognition (Liu et al., 2017; Pei et al., 2016; Wang et al., 2017b), image captioning (Lu et al., 2016; Xu et al.,…”
Spatial Memory for Context Reasoning in Object Detection (2017, background): “It has resemblance to spatial attention, which has been explored in many high-level tasks [47, 68, 91, 92].”
Vision-as-Inverse-Graphics: Obtaining a Rich 3D Explanation of a Scene from a Single Image (2017, methodology): “Recent work by Ren and Zemel [19] also makes use of a LSTM-based recurrent network to direct ﬁxations, and additionally includes a segmentation network for determining segmentation masks of the detected objects.”
SGN: Sequential Grouping Networks for Instance Segmentation (2017, methodology): “However, since RNNs typically do not perform well when the number of time steps is large, these methods have difficulties in scaling up to the challenging multi-object street scenes.”
Weakly Supervised Instance Segmentation Using Class Peak Response (2018, background): “Compared with many fully supervised approaches that typically use complex frameworks including conditional random ﬁelds (CRF) [46, 45], recurrent neural networks (RNN) [30, 32], or template matching [37], to handle instance extraction; our approach is simple yet effective.”
Joint Graph Decomposition and Node Labeling by Local Search (2016, background): “Instance-separating semantic segmentation [5, 6, 15, 22, 23, 24, 31, 32] is the task of assigning, to every point in an image, a label that identifies a class of objects (e.”