High Performance Convolutional Neural Networks for Document Processing

By Kumar Chellapilla, Sidd Puri, Patrice Y. Simard. 2006

706 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “2006 also saw an early GPU-based CNN implementation (Chellapilla et al., 2006); compare earlier GPU-FNNs with a reported speed-up factor of 20 (Oh and Jung, 2004).”
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition (2014, background): “Consequently, there is a strong interest to the task of improving the efﬁciency of this operation (Chintala, 2014; Mathieu et al., 2013; Chellapilla et al., 2006).”
High-Performance Neural Networks for Visual Object Classification (2011, methodology): “CNNs vary in how convolutional and subsampling layers are realized and how the nets are trained.”
Fast Convolutional Nets With fbfft: A GPU Performance Evaluation (2014, background): “3 A popular convolution implementation is to unroll the data until the computation is in the form of a large matrix multiplication (Chellapilla et al. (2006)).”
The hardware lottery (2020, methodology): “In the 2000s, like Edison’s phonograph, GPUs were re-purposed for an entirely unimag-ined use case – to train deep neural networks (Chellapilla et al., 2006; Oh & Jung, 2004; Claudiu Ciresan et al., 2010; Fata-halian et al., 2004; Payne et al., 2005).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “Today’s faster computers, such as GPUs, mitigate the Fundamental Deep Learning Problem for FNNs [181, 34, 198, 38, 40].”
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications (2015, methodology): “When executing convolutions, we apply optimization techniques such as Caffeinated convolution (Chellapilla et al., 2006).”
Scaling Local Self-Attention for Parameter Efficient Visual Backbones (2021, background): “Convolutions have improved over the last decade through changes in (a) the computation of the primitive [4, 25, 34, 52, 56, 28]; (b) the software implementation [5]; (c) the structure of the primitive itself, through for example, grouped convolution [57] and depthwise separable convolution [45].”
Demystifying Parallel and Distributed Deep Learning (2018, methodology): “chnique to transform a discrete convolution into matrix multiplication, using Toeplitz matrices (colloquially known as im2col). The first occurrence of unrolling convolutions in CNNs can be found in [Chellapilla et al. 2006], where CPUs and GPUs were used for training (since the work precedes CUDA, ”
Oblivious Neural Network Predictions via MiniONN Transformations (2017, background): “In practice, for efficiency reasons, convolution is converted into matrix multiplication and addition as well [18], similar to equation 1, except that input and bias vector are matrices: Y :=W · X + B.”
An introduction to computational networks and the computational network toolkit (invited talk) (2014, methodology): “[9] to allow for stacked convolution operations.”
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review (2019, methodology): “The authors implemented the scalable CONV block in a similar fashion to that in [138] as a matrix multiplication by flattening and on-the-fly rearrangement of the feature data.”
DeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications (2017, methodology): “Also, we adopted and improved state-of-the-art matrix multiplication techniques such as model decomposition [31] and unfolding [14] to accelerate multiplication operations (the bottleneck operation in convolutional layers) on mobile GPUs.”
O-CNN (2017, methodology): “In this form, the convolution operation can be converted to a matrix product [Chellapilla et al. 2006; Jia et al. 2014] and computed efficiently on the GPU.”
End-to-End Learning of Energy-Constrained Deep Neural Networks (2018, background): “Tensor convolutions are often converted to matrix-matrix multiplication for computation efficiency reasons [3].”