ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation

By Eduardo Romera, José Manuel Álvarez, Luis Miguel Bergasa, Roberto Arroyo. IEEE transactions on intelligent transportation systems (Print), 2018

1,337 citations recorded by the source corpus.

Semantic segmentation is a challenging task that addresses most of the perception needs of intelligent vehicles (IVs) in an unified way. Deep neural networks excel at this task, as they can be trained end-to-end to accurately classify multiple object categories in an image at pixel level. However, a good tradeoff between high quality and computational resources is yet not present in the state-of-the-art semantic segmentation approaches, limiting their application in real vehicles. In this paper, we propose a deep architecture that is able to run in real time while providing accurate semantic segmentation. The core of our architecture is a novel layer that uses residual connections and factorized convolutions in order to remain efficient while retaining remarkable accuracy. Our approach is able to run at over 83 FPS in a single Titan X, and 7 FPS in a Jetson TX1 (embedded device). A comprehensive set of experiments on the publicly available Cityscapes data set demonstrates that our system achieves an accuracy that is similar to the state of the art, while being orders of magnitude faster to compute than other architectures that achieve top precision. The resulting tradeoff makes our model an ideal approach for scene understanding in IV applications. The code is publicly available at: https://github.com/Eromera/erfnet

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/21b58c8aba44c173493e418a797a1f36c6dae8a9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Seamless Scene Segmentation (2019, neutral): “4% using ERFNet [44] and 66.”
Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection (2020, background): “Although recent advances in deep learning have enabled the use of segmentation networks in real-time [22], various models struggle to to achieve real-time performance.”
RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation (2019, methodology): “This spawned the creation of more efficient approaches such as Bonnet [11], ENet [13], ERFNet [17], and Mobilenets V2 [18], which leverage the law of diminishing returns to find the best trade-off between runtime, the number of parameters, and accuracy.”
Is Context-Aware CNN Ready for the Surroundings? Panoramic Semantic Segmentation in the Wild (2021, background): “ERFNet [33] to strike an optimized efficiency-accuracy trade-”
Deep Dual-Resolution Networks for Real-Time and Accurate Semantic Segmentation of Traffic Scenes (2023, methodology): “Specially, the encoder can be a lightweight backbone pre-trained on Ima-geNet or an efﬁcient variant trained from scratch like ERFNet [5] and ESPNet [14].”
RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes (2019, methodology): “[12] E. Romera, J. M. Álvarez, L. M. Bergasa, and R. Arroyo, “ERFNet: efficient residual factorized convnet for real-time semantic segmentation,” IEEE Trans.”
Moonshine: Distilling with Cheap Convolutions (2017, methodology): “For our teacher network we use an ERFNet (Romera et al., 2017a,b) a 3 × 1 convolution followed by a 1 × 3 convolution.”
ESPNetv2: A Light-Weight, Power Efficient, and General Purpose Convolutional Neural Network (2018, neutral): “5 ERFNet [41] 26 B 69.”
Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth (2019, methodology): “Network architecture We use the ERFNetarchitecture [24] as base-network.”
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis (2020, methodology): “Efficient approaches with custom architectures such as ERFNet [26], LEDNet [27], and ESPNetv2 [32] are quite”
Fast-SCNN: Fast Semantic Segmentation Network (2019, background): “Similar to the object detection [23, 24, 15], speed became one important factor in image segmentation system design [21, 34, 17, 25, 36, 20].”
RESA: Recurrent Feature-Shift Aggregator for Lane Detection (2020, background): “Modern algorithms (Chen and Chen 2017), (Bergasa et al. 2018), (Chen, Liu, and Lian 2019) typically adopt a pixelwise prediction formulation, i.”
CGNet: A Light-Weight Context Guided Network for Semantic Segmentation (2018, methodology): “1The methods involved are Dilation8 [14], DeepLabv2 [15], SQNet [16], ENet [13], PSPNet [17], RefineNet [18], FRRN [19], FCN-8s [4], SegNet [20], ESPNet [12], ERFNet [21], ICNet [11], DenseASPP [10], DeepLabv3+ [9], DFN [8], BiSeNet [22] and the proposed CGNet.”
Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels (2020, background): “The network architecture is the ERFNet encoder-decoder structure [16].”