Spatial As Deep: Spatial CNN for Traffic Scene Understanding

By Xingang Pan, Jianping Shi, Ping Luo, Xiaogang Wang, Xiaoou Tang. 2017

1,226 citations recorded by the source corpus.

Convolutional neural networks (CNNs) are usually built by stacking convolutional operations layer-by-layer. Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored. These relationships are important to learn semantic objects with strong shape priors but weak appearance coherences, such as traffic lanes, which are often occluded or not even painted on the road surface as shown in Fig. 1 (a). In this paper, we propose Spatial CNN (SCNN), which generalizes traditional deep layer-by-layer convolutions to slice-by-slice convolutions within feature maps, thus enabling message passings between pixels across rows and columns in a layer. Such SCNN is particular suitable for long continuous shape structure or large objects, with strong spatial relationship but less appearance clues, such as traffic lanes, poles, and wall. We apply SCNN on a newly released very challenging traffic lane detection dataset and Cityscapse dataset. The results show that SCNN could learn the spatial relationship for structure output and significantly improves the performance. We show that SCNN outperforms the recurrent neural network (RNN) based ReNet and MRF+CNN (MRFNet) in the lane detection dataset by 8.7% and 4.6% respectively. Moreover, our SCNN won the 1st place on the TuSimple Benchmark Lane Detection Challenge, with an accuracy of 96.53%.

Subfields: Automotive Engineering, Civil and Structural Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Civil and Structural Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

VEGA: Towards an End-to-End Configurable AutoML Pipeline (2020, methodology): “We report our VEGA’s results for the lane detection task on the CULane dataset (Xingang et al., 2018) in Table 6.”
Ultra Fast Structure-aware Deep Lane Detection (2020, methodology): “For the Tusimple lane detection benchmark, seven methods are used for comparison, including Res18-Seg [3], Res34-Seg [3], LaneNet [21], EL-GAN [5], SCNN [22] and SAD [9].”
VectorMapNet: End-to-end Vectorized HD Map Learning (2022, methodology): “Most lane detection algorithms (Pan et al., 2018; Neven et al., 2018) use a pixel-level segmentation technique combined with sophisticated post-processing.”
Robust Lane Detection From Continuous Driving Scenes Using Deep Neural Networks (2019, methodology): “In [45], a spatial CNN was proposed to improve the performance of CNN in detecting long continuous shape structures.”
Inter-Region Affinity Distillation for Road Marking Segmentation (2020, positive): “Extensive experiments on three popular datasets (ApolloScape [11], CULane [14] and LLAMAS [1]) show that IntRA-KD consistently outperforms other KD methods, e.”
Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (2018, background): “However, existing works mainly design network architectures to solve the above problems on a single domain, for example, improving scene parsing on the real images of Cityscape dataset [2,20].”
Computing Systems for Autonomous Driving: State of the Art and Challenges (2020, methodology): “To better utilize more visual information of lane markings, SCNN [98] applies a novel convolution operation that aggregates diverse dimension information via processing sliced features and then adds them together.”
End-to-end Lane Shape Prediction with Transformers (2020, methodology): “Existing methods [12, 14, 6, 15, 7] take advantage of the powerful representation capabilities of convolution neural networks (CNNs) to improve the performance of lane detection task by a large margin over traditional methods [11, 13, 20] which are based on hand-crafted features and Hough Transform.”
Learning Lightweight Lane Detection CNNs by Self Attention Distillation (2019, methodology): “In our experiments, we apply the same segmentation losses and augmentation strategy to our method, SCNN, ResNet baselines, and deep supervision methods, to ensure a fair comparison.”
Deep Dual-Resolution Networks for Real-Time and Accurate Semantic Segmentation of Traffic Scenes (2023, background): “It plays a vital role in the environmental perception of intelligent transportation systems and has a wide range of applications such as lane keeping of driver assistant systems and navigation of autonomous vehicles [4], [5].”
Line-CNN: End-to-End Traffic Line Detection With Line Proposal Unit (2020, methodology): “For example, a fundamental task in automatic driving is vehicle navigation or drivable lane recognition, and it is tackled by existing researches from two aspects, namely road region segmentation1 [1]–[5] and traffic line detection [6]–[11] (see Fig.”
CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution (2021, methodology): “Early deep-learning-based methods detect lane lines through segmentation [28, 27].”
PolyLaneNet: Lane Estimation via Deep Polynomial Regression (2020, methodology): “In addition to evaluating on the TuSimple and CULane [7] datasets, the authors provide qualitative results on yet another private US-based dataset.”
RESA: Recurrent Feature-Shift Aggregator for Lane Detection (2020, methodology): “, SCNN (Pan et al. 2018).”
Learning From Synthetic Data for Crowd Counting in the Wild (2019, methodology): “[25] present a spatial encoder via a sequence of convolution on the four directions (down, up, left-to-right and right-to-left).”
CLRNet: Cross Layer Refinement Network for Lane Detection (2022, methodology): “SCNN [17] proposes a message-passing mechanism to address no visual evidence problem, which captures the strong spatial relationship for lanes.”
CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending (2020, methodology): “We conduct neural architecture search on two large lane detection datasets: the CULane [20], and the new CurveLanes dataset.”