PhotoOCR: Reading Text in Uncontrolled Conditions

By Alessandro Bissacco, Mark Joseph Cummins, Yuval Netzer, Hartmut Neven. 2013

511 citations recorded by the source corpus.

We describe Photo OCR, a system for text extraction from images. Our particular focus is reliable text extraction from smartphone imagery, with the goal of text recognition as a user input modality similar to speech recognition. Commercially available OCR performs poorly on this task. Recent progress in machine learning has substantially improved isolated character classification, we build on this progress by demonstrating a complete OCR system using these techniques. We also incorporate modern data center-scale distributed language modelling. Our approach is capable of recognizing text in a variety of challenging imaging conditions where traditional OCR systems fail, notably in the presence of substantial blur, low resolution, low contrast, high image noise and other distortions. It also operates with low latency, mean processing time is 600 ms per image. We evaluate our system on public benchmark datasets for text extraction and outperform all previously reported results, more than halving the error rate on multiple benchmarks. The system is currently in use in many applications at Google, and is available as a user input modality in Google Translate for Android.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Deep Structured Output Learning for Unconstrained Text Recognition (2014, methodology): “Previous methods using CNNs for word recognition (discussed in more detail in section Section 2) has either constrained (Jaderberg et al. (2014b)) or heavily weighted (Bissacco et al. (2013)) the recognition results to be from a dictionary of known words.”
Speeding up Convolutional Neural Networks with Low Rank Expansions (2014, background): “Character classification is an essential part of many text spotting pipelines such as [3, 4, 15, 24, 25, 26, 27, 29, 30, 41, 43].”
Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition (2014, methodology): “Methods of generating candidate character regions include: oriented stroke detection [21], binarization and a sliding window classiﬁer in the PhotoOCR system of [2] (which uses a dictionary of 100k words for re-ranking candidate words), and a CNN character classiﬁer in Alsharif et al. [1] (which sub”
From generic to specific deep representations for visual recognition (2014, neutral): “57 74 v1 [ cs .”
COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images (2016, methodology): “The problem of detecting and recognizing text in scenes has received increasing attention from the computer vision community in recent years [1, 6, 14, 19].”
Arbitrary-Oriented Scene Text Detection via Rotation Proposals (2017, background): “In the past years there have been many attentions in the text detection task [3, 22, 7, 35, 23, 1, 15, 12, 31].”
ASTER: An Attentional Scene Text Recognizer with Flexible Rectification (2019, unknown): “[7] - - - - - 90.”
Text-Attentional Convolutional Neural Network for Scene Text Detection (2015, background): “oped for text component filtering/classification [1], [2], [8], or word/character recognition [2], [8]–[10], by leveraging”
Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes (2018, background): “The character-based recognition methods [2, 22] mostly first localize individual characters and then recognize and group them into words.”
Recursive Recurrent Nets with Attention Modeling for OCR in the Wild (2016, methodology): “Many text recognition methods focus on capturing individual characters of a word as the first step in the system pipeline, and then apply statistical language models or visual structure prediction to refine/prune-out misclassified characters as in [46, 48, 32, 4, 39, 26, 53].”
An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections (2015, background): “Deep neural network-based methods have recently achieved dramatic accuracy improvements in many areas of computer vision, including image classification [23, 45, 25], object detection [11, 33], face recognition [38, 36], and text recognition [2, 20].”
OCR-VQA: Visual Question Answering by Reading Text in Images (2019, background): “, photoOCR, popularly known as scene text recognition [5] and unconstrained handwritten text recognition [15].”
Multi-oriented Text Detection with Fully Convolutional Networks (2016, background): “Most existing approaches are successfully designed for detecting horizontal or near-horizontal text [3, 4, 15, 17, 2, 8, 6, 35, 23, 20, 27].”