FOTS: Fast Oriented Text Spotting with a Unified Network

By Xuebo Liu, Ding Liang, Shipeng Yan, Dagui Chen, Yu Qiao, Junjie Yan. 2018

531 citations recorded by the source corpus.

Incidental scene text spotting is considered one of the most difficult and valuable challenges in the document analysis community. Most existing methods treat text detection and recognition as separate tasks. In this work, we propose a unified end-to-end trainable Fast Oriented Text Spotting (FOTS) network for simultaneous detection and recognition, sharing computation and visual information among the two complementary tasks. Specifically, RoIRotate is introduced to share convolutional features between detection and recognition. Benefiting from convolution sharing strategy, our FOTS has little computation overhead compared to baseline text detection network, and the joint training method makes our method perform better than these two-stage methods. Experiments on ICDAR 2015, ICDAR 2017 MLT, and ICDAR 2013 datasets demonstrate that the proposed method outperforms state-of-the-art methods significantly, which further allows us to develop the first real-time oriented text spotting system which surpasses all previous state-of-the-art results by more than 5% on ICDAR 2015 text spotting task while keeping 22.6 fps.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Human-Computer Interaction

Citations in the corpus

All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting (2019, methodology): “(Liu et al. 2018; Li, Wang, and Shen 2017; He et al. 2018; Busta et al. 2017) share a common idea: text region features are extracted for subsequent recognizer.”
TextCaps: a Dataset for Image Captioning with Reading Comprehension (2020, background): “In recent years, with the availability of large labelled corpora, progress in image captioning has seen steady increase in performance and quality [4,10,12,13,35] and reading scene text (OCR) has matured [8,15,21,23,32].”
Scene Text Visual Question Answering (2019, methodology): “Approaches that use Connectionist Temporal Classification have also been widely used in scene text recognition, in works such as [47, 7, 57, 12, 38], among others.”
Character Region Awareness for Text Detection (2019, background): “Recently, scene text detectors based on deep learning have shown promising performance [8, 40, 21, 4, 11, 10, 12, 13, 17, 24, 25, 32, 26].”
Shape Robust Text Detection With Progressive Scale Expansion Network (2018, methodology): “For the regression-based approaches [36, 43, 32, 16, 42, 23, 11, 13, 27], the text targets are usually represented in the forms of rectangles or quadrangles with certain orientations.”
Structured Multimodal Attentions for TextVQA (2020, background): “Some recent works are also proposed to address both text detection and recognition in one framework [40], [41], [42], [43].”
Rosetta: Large Scale System for Text Detection and Recognition in Images (2018, methodology): “Following Jaderberg’s work, many new methods have recently appeared that use CNN architectures to detect words in images, including [16, 21, 25, 32].”
Arbitrary Shape Scene Text Detection With Adaptive Text Region Representation (2019, methodology): “Because test image scale has a deep impact on the detection results, such as FOTS [15] uses different scales for different datasets, we also use different test scales for different datasets for best performance.”
TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network (2018, methodology): “Moreover, to handle oriented text regions, rotational text proposal [5] and RoI-Rotate [30] have been developed to address this issue using affine transformation, motivated by the idea of Spatial Transformer Network [21] to learn a transformation matrix.”
ABCNet: Real-Time Scene Text Spotting With Adaptive Bezier-Curve Network (2020, methodology): “, RoI Pooling [19], RoIRotate [25], Text-Align-Sampling [11], or RoI Transform [37] cannot properly align features of oriented or curved text (RoISlide [44] numerous predicting segments).”
Multimodal Attention with Image Text Spatial Relationship for OCR-Based Image Captioning (2020, background): “[23] improve the model via a better text detection algorithm which has the ability to deal with more complicated situations.”
Arbitrary-Oriented Object Detection with Circular Smooth Label (2020, background): “In particular, rotation detection has played a huge role in the field of aerial images [2, 4, 41, 42, 44], scene text [12, 18, 19, 24, 27, 49] and face [11, 33, 34].”
Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss (2021, background): “Arbitrary-oriented objects are ubiquitous for detection across visual datasets, such as aerial images [65, 2, 8, 69], scene text [82, 31, 20, 35, 27], faces [47] and 3D objects [78], retail scenes [5, 41], etc.”
Scene Text Detection and Recognition: The Deep Learning Era (2018, background): “(d) [45], [81] jointly train a detection model and recognition model.”
R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object (2019, methodology): “78 720p Titan X 24 FOTS [28] – 91.”
MMRotate: A Rotated Object Detection Benchmark using PyTorch (2022, methodology): “In light of this, deep learning has also been applied to areas such as faces detection [27], text detection [12, 13, 17, 20, 45] and aerial images detection [32, 35, 38].”
Learning Modulated Loss for Rotated Object Detection (2019, methodology): “For ICDAR2015, there are rich existing methods such as RCNN, Deep direct regression [25] and FOTS [53], and the current state-of-art has reached 91.”
Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things (2020, methodology): “Deep learning has advanced this area by learning more representative feature [87], devising better representation of text proposals [88], and using large-scale synthetic dataset [89].”
Dense Label Encoding for Boundary Discontinuity Free Rotation Detection (2020, methodology): “RRPN [26], TextBoxes++ [18] and RRD [19] and FOTS [23] are some advanced methods for scene text detection.”