Recursive Recurrent Nets with Attention Modeling for OCR in the Wild

By Chen-Yu Lee, Simon Osindero. Computer Vision and Pattern Recognition, 2016

490 citations recorded by the source corpus.

We present recursive recurrent neural networks with attention modeling (R2AM) for lexicon-free optical character recognition in natural scene images. The primary advantages of the proposed method are: (1) use of recursive convolutional neural networks (CNNs), which allow for parametrically efficient and effective image feature extraction, (2) an implicitly learned character-level language model, embodied in a recurrent neural network which avoids the need to use N-grams, and (3) the use of a soft-attention mechanism, allowing the model to selectively exploit image features in a coordinated way, and allowing for end-to-end training within a standard backpropagation framework. We validate our method with state-of-the-art performance on challenging benchmark datasets: Street View Text, IIIT5k, ICDAR and Synth90k.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a8c9d85147039ca54b0439cde05ef8c33efecf00

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Xiang Bai
2. Chunhua Shen
3. Cong Yao
4. Luc Van Gool
5. Yu Qiao
6. Dacheng Tao
7. Xiaogang Wang
8. Minghui Liao

Citations in the corpus

• ESIR: End-To-End Scene Text Recognition via Iterative Image Rectification (2018, background): “For example, [21] learns broader contextual information and uses an attention based decoder for sequence generation.”
• Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes (2018, methodology): “Besides, different from the previous sequence-based recognition methods [45, 44, 26] which are designed for 1-D sequence, we recognize text via semantic”
• Decoupled Attention Network for Text Recognition (2019, background): “(Lee and Osindero 2016) and (Shi et al. 2016) introduced attention mechanism to automatically align and translate words.”
• Global Weighted Average Pooling Bridges Pixel-level Localization and Image-level Classification (2018, methodology): “Comparision with the soft attention model: Soft attention model (SAM) is proposed by Bahdanau [37] in neural machine translation area, and has been successfully used for computer vision tasks [38], [39], [40], [41], [42], [43], [44].”
• Attention, please! A survey of neural attention models in deep learning (2021, unknown): “(2018), Lee and Osindero (2016), Chu et al.”
• TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network (2018, methodology): “n an end-to-end trained model [45][31]. The state-of-the-art text recognition methods can be formulated as an end-to-end "translation" problem from a cropped text image to text sequence [38][24]. More recently, in contrast to multi-stage text reading systems, fully trainable models to tack”
• Symmetry-Constrained Rectification Network for Scene Text Recognition (2019, methodology): “Deep learning based methods [20, 17, 25, 48, 45, 44, 7, 29, 3, 8, 46, 57] have been dominating this area in recent years.”
• ASTER: An Attentional Scene Text Recognizer with Flexible Rectification (2019, background): “The conference version of this paper [55] was the first, in parallel with [36], to explore such models in the task of scene text recognition.”
• What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis (2019, methodology): “RCNN is a variant of CNN that can be applied recursively to adjust its receptive fields depending on the character shapes [15, 26].”
• Neural Machine Translation: A Review (2019, background): “recognition [84, 85], cross-lingual word-to-phone alignment [86], bioinformatics [87], text summarization [88], text normalization [89], grammatical error correction [90], question answering [91–93], natural language understanding and inference [62, 94–96], uncertainty detection [97], photo optical ”
• Bidirectional Decoder Networks for Attention-Based End-to-End Offline Handwriting Recognition (2016, methodology): “However, many successes with convolutional operations and subsampling suggest that it might be beneficial to produce less observations than output symbols and to decompress the output sequence using the total set of generated features [9].”
• An End-to-End TextSpotter with Explicit Alignment and Attention (2018, background): “Recently, attention mechanism has been developed for word recognition [21, 1], where an implicit attention is learned automatically to enhance deep features in the decoding process.”
• Detecting Curve Text in the Wild: New Dataset and New Solution (2017, methodology): “The recent popular recognition methods [29, 28, 1, 19] all require single row text line in each bounding box.”
• RobustScanner: Dynamically Enhancing Positional Clues for Robust Text Recognition (2020, methodology): “Later, top-down methods [12,40,15,11,24,41,3,42,57,8,7,25,50,35,1,55,45,16] were proposed, in which text sequences are end-to-end predicted without the single character detection.”
• Towards End-to-End Text Spotting with Convolutional Recurrent Neural Networks (2017, methodology): “Our model is motivated by recent progresses in DNN models such as Faster R-CNN [21] and sequence-tosequence learning [24, 16], but we take the special characteristics of text into consideration.”
• Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition (2018, methodology): “For regular text, it is presented in (Lee and Osindero 2016; Shi et al. 2016) that the 1 D attention based encoder-decoder framework is able to align between input subsequences and decoded characters.”