Multi-oriented Scene Text Detection via Corner Localization and Region Segmentation

By Pengyuan Lyu, Cong Yao, Shuicheng Yan, Xiang Bai. 2018

339 citations recorded by the source corpus.

Previous deep learning based state-of-the-art scene text detection methods can be roughly classified into two categories. The first category treats scene text as a type of general objects and follows general object detection paradigm to localize scene text by regressing the text box locations, but troubled by the arbitrary-orientation and large aspect ratios of scene text. The second one segments text regions directly, but mostly needs complex post processing. In this paper, we present a method that combines the ideas of the two types of methods while avoiding their shortcomings. We propose to detect scene text by localizing corner points of text bounding boxes and segmenting text regions in relative positions. In inference stage, candidate boxes are generated by sampling and grouping corner points, which are further scored by segmentation maps and suppressed by NMS. Compared with previous methods, our method can handle long oriented text naturally and doesn't need complex post processing. The experiments on ICDAR2013, ICDAR2015, MSRA-TD500, MLT and COCO-Text demonstrate that the proposed algorithm achieves better or comparable results in both accuracy and efficiency. Based on VGG16, it achieves an F-measure of 84.3% on ICDAR2015 and 81.5% on MSRA-TD500.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Chunhua Shen
2. Xiang Bai
3. Luc Van Gool
4. Dacheng Tao
5. Cong Yao
6. Ming-Hsuan Yang
7. Yu Qiao
8. Xiaogang Wang

In Media Technology

1. Cong Yao
2. Xiang Bai
3. Minghui Liao
4. Xin He
5. Shangbang Long
6. Pengyuan Lyu
7. Zhaoyi Wan
8. Guisong Xia

Citations in the corpus

• All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting (2019, methodology): “Following (Lyu et al. 2018b), our model is not trained with the training set of COCO-Text.”
• Real-Time Scene Text Detection With Differentiable Binarization and Adaptive Scale Fusion (2022, methodology): “Following the previous methods [37], [39], [65], we include extra 400 training images from HUSTTR400 [58].”
• A Survey of Deep Learning-Based Object Detection (2019, background): “[271] combine the ideas of the two categories above avoiding their shortcomings by locating corner points”
• Efficient and Accurate Arbitrary-Shaped Text Detection With Pixel Aggregation Network (2019, methodology): “Following [42, 37, 35], we pre-train our model on this dataset.”
• Scene Text Detection and Recognition: The Deep Learning Era (2018, methodology): “(b) PixelLink [20]: predict for each pixel, text/non-text classification and whether it belongs to the same text as adjacent pixels or not/ (c) Corner Localization [89]: predict the four corners of each text and group those belonging to the same text instances.”
• Character Region Awareness for Text Detection (2019, background): “Recently, scene text detectors based on deep learning have shown promising performance [8, 40, 21, 4, 11, 10, 12, 13, 17, 24, 25, 32, 26].”
• Gliding Vertex on the Horizontal Bounding Box for Multi-Oriented Object Detection (2019, methodology): “Some other methods [9], [10], [11] alternatively detect horizontal object parts followed by a grouping process.”
• Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes (2018, background): “A large number of methods have been proposed to detect scene text [7, 36, 37, 50, 19, 23, 54, 21, 47, 54, 56, 30, 52, 55, 34, 15, 48, 43, 57, 16, 35, 31].”
• TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes (2018, methodology): “Following previous works [47, 22], we also include the images”
• Object Detection in 20 Years: A Survey (2019, methodology): “range texts, where a group of corners or a closed boundary corresponds to an individual line of text [337, 359, 360].”
• Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes (2019, methodology): “As shown in Table 7, LOMO has the performance leading in single-scale testing compared with most of existing methods [9, 23, 36, 41], expect for Border [36] and AFN-RPN [41] that do not indicate which testing scales are used.”
• Structured Multimodal Attentions for TextVQA (2020, background): “Most works address it separately by designing high performance text detector [31], [32], [33], [34] and sophisticated word recognizer [35], [36], [37], [38], [39], which can be combined together to obtain text contents in an image finally.”
• Real-time Scene Text Detection with Differentiable Binarization (2019, methodology): “Text border is used in (Xue, Lu, and Zhan 2018) to split the text instances, Mask TextSpotter (Lyu et al. 2018a; Liao et al. 2019) detected arbitrary-shape text instances in an instance segmentation manner based on Mask R-CNN.”
• Symmetry-Constrained Rectification Network for Scene Text Recognition (2019, background): “Scene text reading [60, 32, 59, 54, 53, 36, 35] is an important, active research area in computer vision, which can be applied to a wide range of real-world applications, such as self-driving cars, assistant position systems, and guide board recognition [43].”