Deep Direct Regression for Multi-oriented Scene Text Detection

By Wenhao He, Xu-Yao Zhang, Fei Yin, Chenglin Liu. 2017

388 citations recorded by the source corpus.

In this paper, we first provide a new perspective to divide existing high performance object detection methods into direct and indirect regressions. Direct regression performs boundary regression by predicting the offsets from a given point, while indirect regression predicts the offsets from some bounding box proposals. In the context of multioriented scene text detection, we analyze the drawbacks of indirect regression, which covers the state-of-the-art detection structures Faster-RCNN and SSD as instances, and point out the potential superiority of direct regression. To verify this point of view, we propose a deep direct regression based method for multi-oriented scene text detection. Our detection framework is simple and effective with a fully convolutional network and one-step post processing. The fully convolutional network is optimized in an end-to-end way and has bi-task outputs where one is pixel-wise classification between text and non-text, and the other is direct regression to determine the vertex coordinates of quadrilateral text boundaries. The proposed method is particularly beneficial to localize incidental scene texts. On the ICDAR2015 Incidental Scene Text benchmark, our method achieves the F-measure of 81%, which is a new state-ofthe-art and significantly outperforms previous approaches. On other standard datasets with focused scene texts, our method also reaches the state-of-the-art performance.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting (2019, background): “In (Liu et al. 2018; Li, Wang, and Shen 2017; He et al. 2018; Busta et al. 2017; Liu and Jin 2017; He et al. 2017), the detected bounding box of each text instance is represented with a rectangle, which only can tightly cover straight text instances.”
Scene Text Visual Question Answering (2019, methodology): “Several works have been proposed addressing text detection such as [36, 35, 60, 21] which are mostly based on Fully Convolutional Neural Networks.”
TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network (2018, background): “[17] propose to directly regress multi-oriented text in quadrangle representation.”
Detecting Curve Text in the Wild: New Dataset and New Solution (2017, background): “And in 2017, plentiful quadrilateral based detection methods [21, 27, 38, 10, 4] emerged.”
Efficient and Accurate Arbitrary-Shaped Text Detection With Pixel Aggregation Network (2019, methodology): “EAST [58] and DeepReg [17] adopt FCNs to predict shrinkable text score maps and perform per-pixel regression, followed by a post-processing NMS. TextSnake [35] models text instances with ordered disks and text center lines, which is able to represent text in arbitrary shapes.”
FOTS: Fast Oriented Text Spotting with a Unified Network (2018, methodology): “65 - NLPR-CASIA [15] 82 80 81 Stradvision-2 [26] 43.”
R2CNN: Rotational Region CNN for Orientation Robust Scene Text Detection (2017, methodology): “Deep direct regression [34] is proposed for multi-oriented scene text detection.”
Multi-oriented Scene Text Detection via Corner Localization and Region Segmentation (2018, methodology): “Compared with regressing text bounding box from anchors ( [28, 33]) or from text regions ([57, 17]), the yielded bounding boxes are more accurate, particularly for long text.”
Total-Text: toward orientation robustness in scene text detection (2019, background): “[41] demonstrated the potential edge of direct regression over offset prediction with a single fully convolutional network.”
Shape Robust Text Detection With Progressive Scale Expansion Network (2018, neutral): “nk [32] X 73.1 76.8 75.0 - SSTD [11] X 80.23 73.86 76.91 7.7 WordSup [13] X 79.33 77.03 78.16 - EAST [43] - 83.57 73.47 78.2 13.2 RRPN [28] - 82.0 73.0 77.0 - R2CNN [16] - 85.62 79.68 82.54 - DeepReg [12] - 82.0 80.0 81.0 - PixelLink [4] - 82.9 81.7 82.3 7.3 Lyu et al. [27] X 94.1 70.7 80.7 3.6 RRD ”
Rotation-Sensitive Regression for Oriented Scene Text Detection (2018, background): “The works in [11, 26] are such examples; 2) Multi-oriented text detectors [48, 50, 51, 38, 29, 13, 12, 14, 46] which focus on detecting text of arbitrary orientations.”
Robust Classification with Convolutional Prototype Learning (2018, background): “In recent years, convolutional neural networks [16] (CNNs) have achieved great success for pattern recognition and computer vision, leading to important progress in a variety of tasks, like image classification [15, 8, 7, 34], object detection [1, 29, 9], instance segmentation [28, 6] and so on.”
Real-time Scene Text Detection with Differentiable Binarization (2019, methodology): “SSTD (He et al. 2017a) proposed an attention mechanism to roughly identi-ﬁes text regions.”
Scene Text Detection and Recognition: The Deep Learning Era (2018, methodology): “In text instance level methods [18], [46], [59], [73], [74], [82], [90], [119], [163], [168], detection of text follows the standard routine of general object detection, where a regionproposal network and a refinement network are combined to make predictions.”
R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object (2019, neutral): “s. Device FPS CTPN [37] – 51.56 74.22 60.85 – – – SegLink [36] – 76.80 73.10 75.00 – – – RRPN [30] – 82.17 73.23 77.44 – – <1 EAST [44] – 78.33 83.27 80.72 720p Titan X 13.2 Deep direct regression [14] –80.00 82.00 81.00 <1 R2CNN [17] – 79.68 85.62 82.54 720p K80 0.44 FOTS RT [28] – 85.95 79.8”
TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes (2018, background): “EAST [47] and Deep Regression [8] both directly produce the rotated boxes or quadrangles of text, in a per-pixel manner.”