Improving patch-based scene text script identification with ensembles of conjoined networks

By Lluis Gomez i Bigorda, Anguelos Nicolaou, Dimosthenis Karatzas. 2016

79 citations recorded by the source corpus.

We present a patch-based classification method for script identificattion in the wild.We describe a novel method based on the use of ensembles of conjoined networks (ECN).The ECN learns discriminative local features and their relative importance in a global classification rule.Our experiments demonstrate state-of-the-art results in three script identification datasets. This paper focuses on the problem of script identification in scene text images. Facing this problem with state of the art CNN classifiers is not straightforward, as they fail to address a key characteristic of scene text instances: their extremely variable aspect ratio. Instead of resizing input images to a fixed aspect ratio as in the typical use of holistic CNN classifiers, we propose here a patch-based classification framework in order to preserve discriminative parts of the image that are characteristic of its class.We describe a novel method based on the use of ensembles of conjoined networks to jointly learn discriminative stroke-parts representations and their relative importance in a patch-based classification scheme. Our experiments with this learning procedure demonstrate state-of-the-art results in two public script identification datasets.In addition, we propose a new public benchmark dataset for the evaluation of multi-lingual scene text end-to-end reading systems. Experiments done in this dataset demonstrate the key role of script identification in a complete end-to-end system that combines our script identification method with a previously published text detector and an off-the-shelf OCR engine.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jianfei Cai
2. G. Wang
3. Jiuxiang Gu
4. Amir Shahroudy
5. Tsuhan Chen
6. Chenglin Liu
7. Bing Shuai
8. Fei Yin

In Signal Processing

1. Jianfei Cai
2. Chenglin Liu
3. Dimosthenis Karatzas
4. Gang Wang
5. Jiuxiang Gu
6. Bing Shuai
7. Amir Shahroudy
8. Tsuhan Chen

Citations in the corpus

• Recent advances in convolutional neural networks (2015, background): “Recently, the focus has been shifted to text recognition on scene images due to the growing trend of high-level visual understanding in computer vision research [233, 234].”
• ICDAR2017 Robust Reading Challenge on Multi-Lingual Scene Text Detection and Script Identification - RRC-MLT (2017, methodology): “7) Synthetic-ECN: Authors: Yizhi Wang Affiliation: Peking University Method name: A Synthetic-ecn method Method description: The ECN-based model [10] is used.”