Detecting Text in Natural Image with Connectionist Text Proposal Network

By Zhi Tian, Weilin Huang, Tong He, Pan He, Yu Qiao. European Conference on Computer Vision, 2016

1,005 citations recorded by the source corpus.

We propose a novel Connectionist Text Proposal Network (CTPN) that accurately localizes text lines in natural image. The CTPN detects a text line in a sequence of fine-scale text proposals directly in convolutional feature maps. We develop a vertical anchor mechanism that jointly predicts location and text/non-text score of each fixed-width proposal, considerably improving localization accuracy. The sequential proposals are naturally connected by a recurrent neural network, which is seamlessly incorporated into the convolutional network, resulting in an end-to-end trainable model. This allows the CTPN to explore rich context information of image, making it powerful to detect extremely ambiguous text. The CTPN works reliably on multi-scale and multi-language text without further post-processing, departing from previous bottom-up methods requiring multi-step post filtering. It achieves 0.88 and 0.61 F-measure on the ICDAR 2013 and 2015 benchmarks, surpassing recent results [8, 35] by a large margin. The CTPN is computationally efficient with 0.14 s/image, by using the very deep VGG16 model [27]. Online demo is available: http://textdet.com/.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b620548e06b03e3cc2e9e775c39f5b3d5a4eb19a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Xiang Bai
2. Cong Yao
3. Chunhua Shen
4. Dacheng Tao
5. Luc Van Gool
6. Yu Qiao
7. Xiaogang Wang
8. Andrew Zisserman

Citations in the corpus

• MovieNet: A Holistic Dataset for Movie Understanding (2020, methodology): “Given a shot, we first use a scene text detector [69] to detect the text on each frame.”
• A Fast Uyghur Text Detector for Complex Background Images (2018, methodology): “2) Evaluation of Text Localization: Text localization results on the ICDAR 2011 dataset are reported in Table III, where the methods in [18] and [19] are on the basis of deep learning.”
• FOTS: Fast Oriented Text Spotting with a Unified Network (2018, methodology): “[49] employ a vertical anchor mechanism to predict the fixedwidth sequential proposals and then connect them.”
• Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes (2018, background): “A large number of methods have been proposed to detect scene text [7, 36, 37, 50, 19, 23, 54, 21, 47, 54, 56, 30, 52, 55, 34, 15, 48, 43, 57, 16, 35, 31].”
• TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes (2018, neutral): “ults of dierent methods on ICDAR 2015. stands for multiscale, yindicates that the base net of the model is not VGG16. Method Precision Recall F-measure FPS Zhang et al. [27] 70.8 43.0 53.6 0.48 CTPN [44] 74.2 51.6 60.9 7.1 Yao et al. [1] 72.3 58.7 64.8 1.61 SegLink [2] 73.1 76.8 75.0 - EAST [3] 80.5”
• TextBoxes++: A Single-Shot Oriented Scene Text Detector (2018, methodology): “However, the proposed method in [33] has limited adaptability for oriented text due to the single orientation of the Bidirectional Long Short-Term Memory (BLSTM) [43], and the work in [39] has two super parameters for linking the segments, which are determined by grid search and difficult to adjust.”
• Object Detection in 20 Years: A Survey (2019, background): “To improve broken and blurred text detection: A recent idea to deal with broken and blurred texts is to use word level [77, 361] recognition and sentence level recognition [335].”
• An End-to-End TextSpotter with Explicit Alignment and Attention (2018, background): “cent deep learning technologies, as substantiated by recent works [17, 8, 2, 23, 12, 35, 43, 31, 33, 22, 26].”
• OCR-Free Document Understanding Transformer (2021, background): “Recently, focusing on the homogeneity and locality of texts, component-level approaches were proposed [56,4].”
• Multi-oriented Scene Text Detection via Corner Localization and Region Segmentation (2018, neutral): “7%, which surpasses all competitors [56, 47, 54, 40, 57, 16] .”
• Detecting Curve Text in the Wild: New Dataset and New Solution (2017, methodology): “For DMPNet [21] and CTPN [32], we both use Caffe [15] framework to re-implement these methods.”
• EAST: An Efficient and Accurate Scene Text Detector (2017, methodology): “[34]; (e) Our pipeline, which eliminates most intermediate steps, consists of only two stages and is much simpler than previous solutions.”
• Rotation-Sensitive Regression for Oriented Scene Text Detection (2018, methodology): “Based on the representation of detection output, scene text detectors could be roughly divided into two categories: 1) Horizontal text detectors [49, 19, 11, 11, 44, 26] which detect words or text lines in terms of horizontal bounding boxes in the same way as general object detectors.”
• Detecting Oriented Text in Natural Images by Linking Segments (2017, methodology): “Our method is similar in spirit to a recent work [22], which detects text lines by finding and grouping a sequence of fine-scale text proposals through a CNN coupled with recurrent neural layers.”
• PixelLink: Detecting Scene Text via Instance Segmentation (2018, background): “To further check the generalization ability of PixelLink, the same 2s model is also tested on COCO-Text evaluation set without any fine-tuning, achieving a performance of 35.4, 54.0, 42.4 for recall, precision and F-score, exceeding the corresponding results 32.4, 50.4, 39.5 of EAST (Zhou et al. 2017”
• R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object (2019, neutral): “CTPN [37] – 51.”