Detecting Oriented Text in Natural Images by Linking Segments

By Baoguang Shi, Xiang Bai, Serge J. Belongie. Computer Vision and Pattern Recognition, 2017

693 citations recorded by the source corpus.

Most state-of-the-art text detection methods are specific to horizontal Latin text and are not fast enough for real-time applications. We introduce Segment Linking (SegLink), an oriented text detection method. The main idea is to decompose text into two locally detectable elements, namely segments and links. A segment is an oriented box covering a part of a word or text line, A link connects two adjacent segments, indicating that they belong to the same word or text line. Both elements are detected densely at multiple scales by an end-to-end trained, fully-convolutional neural network. Final detections are produced by combining segments connected by links. Compared with previous methods, SegLink improves along the dimensions of accuracy, speed, and ease of training. It achieves an f-measure of 75.0% on the standard ICDAR 2015 Incidental (Challenge 4) benchmark, outperforming the previous best by a large margin. It runs at over 20 FPS on 512x512 images. Moreover, without modification, SegLink is able to detect long lines of non-Latin text, such as Chinese.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/0016d247423bede7f66224044ccc08f21d49fd39

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Xiang Bai
2. Cong Yao
3. Chunhua Shen
4. Luc Van Gool
5. Dacheng Tao
6. Yu Qiao
7. Xiaogang Wang
8. Ming-Hsuan Yang

In Media Technology

1. Cong Yao
2. Xiang Bai
3. Minghui Liao
4. Xue Yang
5. Xin He
6. Shangbang Long
7. Junchi Yan
8. Lianwen Jin

Citations in the corpus

• PixelLink: Detecting Scene Text via Instance Segmentation (2018, methodology): “Locations of text instances, or their segments/slices, are predicted as offsets from reference boxes ( e.g. , TextBoxes, SegLink, CTPN), or absolute locations of bounding boxes ( e.g. , EAST).”
• Real-time Scene Text Detection with Differentiable Binarization (2019, background): “TextBoxes (Liao et al. 2017), TextBoxes++ (Liao, Shi, and Bai 2018), SegLink (Shi, Bai, and Belongie 2017), and RRD (Liao et al. 2018) achieved fast text detection by following the detection architecture of SSD (Liu et al. 2016).”
• Multi-oriented Scene Text Detection via Corner Localization and Region Segmentation (2018, background): “7%, which surpasses all competitors [56, 47, 54, 40, 57, 16] .”
• FOTS: Fast Oriented Text Spotting with a Unified Network (2018, methodology): “Reading text in natural images has attracted increasing attention in the computer vision community [49, 43, 53, 44, 14, 15, 34], due to its numerous practical applications in document analysis, scene understanding, robot navigation, and image retrieval.”
• Real-Time Scene Text Detection With Differentiable Binarization and Adaptive Scale Fusion (2022, methodology): “TextBoxes [25], TextBoxes+ + [24], SegLink [43], and RRD [27] achieved fast text detection by following the detection architecture of SSD [30].”
• Arbitrary Shape Scene Text Detection With Adaptive Text Region Representation (2019, methodology): “Recall Precision Hmean SegLink [24] 40 scale input image with single model, only the results generated in this situation are used here.”
• Single Shot Text Detector with Regional Attention (2017, background): “el. Furthermore, accurate segmentation of text from a predicted heatmap is complicated, and often requires a number of heuristic post-processing steps. Our work also relates to most recent approaches [18,28, 16,4,37,24] which are extended from the state-of-the-art object detectors, such as SSD [17] ”
• Scene Text Detection and Recognition: The Deep Learning Era (2018, methodology): “At the same time, researchers from different viewpoints are enriching and promoting the community into more in-depth work, aiming at different targets, e.g. faster and simpler pipeline (Zhou et al. 2017), text of varying aspect ratios (Shi et al. 2017a), and synthetic data (Gupta et al. 2016).”
• Rotation-Sensitive Regression for Oriented Scene Text Detection (2018, methodology): “The works in [11, 26] are such examples; 2) Multi-oriented text detectors [48, 50, 51, 38, 29, 13, 12, 14, 46] which focus on detecting text of arbitrary orientations.”
• Detecting Curve Text in the Wild: New Dataset and New Solution (2017, methodology): “Note that for East [38] and Seglink [27], we re-implement these methods based on the unofficial source codes from github, and for fair comparison, we do not use huge synthetic data to pre-train seglink which may reduce its performance.”
• Arbitrary-Oriented Scene Text Detection via Rotation Proposals (2017, neutral): “6 s hust orientedText [59] 77 75 76 CTPN [42] 93 83 88 RRPN 82 68 74 0.”
• Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes (2019, methodology): “Component-based methods [38, 34, 29, 30, 10, 33, 19] first detect individual text parts or characters, and then group them into words with a set of post-processing steps.”
• Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition (2017, background): “In fact, both of these algorithms, along with [7], have reported their failure on the same curved text images in MSRA-TD500 as illustrated in Fig.”
• Cascaded Segmentation-Detection Networks for Text-Based Traffic Sign Detection (2018, background): “[43] decompose text into segments and links, where a segment is an oriented box covering a part of a word or text line, and a link is used to connect two segments belonging to the same word or text line.”
• TextBoxes++: A Single-Shot Oriented Scene Text Detector (2018, methodology): “[39] propose to detect text parts and then link them together.”