End-to-end Lane Shape Prediction with Transformers

By Ruijin Liu, Zejian Yuan, Tie Liu, Zhiliang Xiong. IEEE Workshop/Winter Conference on Applications of Computer Vision, 2020

353 citations recorded by the source corpus.

Lane detection, the process of identifying lane markings as approximated curves, is widely used for lane departure warning and adaptive cruise control in autonomous vehicles. The popular pipeline that solves it in two steps— feature extraction plus post-processing, while useful, is too inefficient and flawed in learning the global context and lanes’ long and thin structures. To tackle these issues, we propose an end-to-end method that directly outputs parameters of a lane shape model, using a network built with a transformer to learn richer structures and context. The lane shape model is formulated based on road structures and camera pose, providing physical interpretation for parameters of network output. The transformer models non-local interactions with a self-attention mechanism to capture slender structures and global context. The proposed method is validated on the TuSimple benchmark and shows state-of-the-art accuracy with the most lightweight model size and fastest speed. Additionally, our method shows excellent adaptability to a challenging self-collected lane detection dataset, showing its powerful deployment potential in real applications. Codes are available at https://github.com/liuruijin17/LSTR.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/566f12bc938fab439ca3a1f6162f54be02abddf2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Tao A. Xiang
3. Ling Shao
4. Yanwei Fu
5. Li Zhang
6. Philip H. S. Torr
7. Yunhe Wang
8. Xinggang Wang

In Automotive Engineering

1. Xiaohao Chen
2. Philip H. S. Torr
3. Dacheng Tao
4. Tao A. Xiang
5. Ping Tan
6. Lizhe Liu
7. Siyu Zhu
8. Nian Liu

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Ling Shao
4. Yi Yang
5. Ming-Hsuan Yang
6. Xiaogang Wang
7. Philip H. S. Torr
8. Jiashi Feng

Citations in the corpus

• A Survey on Vision Transformer (2020, background): “[81] propose to improve performance of curve lane detection by learning the global context with transformer network.”
• CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution (2021, methodology): “Another approach is anchor-based methods [25, 22, 39].”
• Visual Grounding with Transformers (2021, methodology): “Our model is inspired by the recent achievements of Transformers in both natural language processing [38] and computer vision [11, 39, 20, 3, 53, 8, 27].”
• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers (2020, background): “STTR [33] and LSTR [35] adopt transformer for disparity estimation and lane shape prediction respectively.”
• CLRNet: Cross Layer Refinement Network for Lane Detection (2022, methodology): “LSTR [13] takes road structures with camera pose into account to model the lane shape, then introduces the trans-former to lane detection task to get the global feature.”
• Visual Saliency Transformer (2021, background): “Some works combined CNNs and transformers into hybrid architectures for object detection [3, 91], panoptic segmentation [62], lane shape prediction [40], and so on.”
• MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction (2022, methodology): “LSTR (Liu et al., 2021a) adopts the Trans-former architecture to directly output parameters of a lane shape model.”