TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network

By Yipeng Sun, Chengquan Zhang, Zuming Huang, Jiaming Liu, Junyu Han, Errui Ding. 2018

66 citations recorded by the source corpus.

Reading text from images remains challenging due to multi-orientation, perspective distortion and especially the curved nature of irregular text. Most of existing approaches attempt to solve the problem in two or multiple stages, which is considered to be the bottleneck to optimize the overall performance. To address this issue, we propose an end-to-end trainable network architecture, named TextNet, which is able to simultaneously localize and recognize irregular text from images. Specifically, we develop a scale-aware attention mechanism to learn multi-scale image features as a backbone network, sharing fully convolutional features and computation for localization and recognition. In text detection branch, we directly generate text proposals in quadrangles, covering oriented, perspective and curved text regions. To preserve text features for recognition, we introduce a perspective RoI transform layer, which can align quadrangle proposals into small feature maps. Furthermore, in order to extract effective features for recognition, we propose to encode the aligned RoI features by RNN into context information, combining spatial attention mechanism to generate text sequences. This overall pipeline is capable of handling both regular and irregular cases. Finally, text localization and recognition tasks can be jointly trained in an end-to-end fashion with designed multi-task loss. Experiments on standard benchmarks show that the proposed TextNet can achieve state-of-the-art performance, and outperform existing approaches on irregular datasets by a large margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chunhua Shen
2. Hao Chen
3. Xiang Bai
4. Yuliang Liu
5. Chenglin Liu
6. Tong He
7. Errui Ding
8. Lianwen Jin

In Computer Vision and Pattern Recognition

1. Xiang Bai
2. Chunhua Shen
3. Cong Yao
4. Luc Van Gool
5. Tong He
6. Lianwen Jin
7. Yu Qiao
8. Yuliang Liu

Citations in the corpus

• All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting (2019, background): “Methods in (Liu et al. 2018; He et al. 2018; Busta et al. 2017; Sun et al. 2018) extract text features in the minimum oriented rectangular box, which can tackle oriented text.”
• Total-Text: toward orientation robustness in scene text detection (2019, background): “[51] proposed an end-to-end trainable text spotting frameworkwhich is robust against text of all orientations.”
• Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes (2019, methodology): “However, we use RoI transform layer [32] to extract the feature block of the input text quadrangle instead of RoI pooling [29] layer or RoI align [5] layer.”
• ABCNet: Real-Time Scene Text Spotting With Adaptive Bezier-Curve Network (2020, methodology): “, RoI Pooling [19], RoIRotate [25], Text-Align-Sampling [11], or RoI Transform [37] cannot properly align features of oriented or curved text (RoISlide [44] numerous predicting segments).”