Global training of document processing systems using graph transformer networks

By L. Eon Bottou, Yoshua Bengio, Yann LeCun. 1997

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Natural Language Processing (Almost) from Scratch (2011, background): “Remark 3 (Graph Transformer Networks) Our approach is a particular case of the discriminative forward training for graph transformer networks (GTNs) (Bottou et al., 1997; Le Cun et al., 1998).”
Neural Module Networks (2015, background): “Other policy- and algorithm-learning approaches with modular substructure include [17, 5, 4].”
Sequence Transduction with Recurrent Neural Networks (2012, methodology): “Section 2 defines the RNN transducer, showing how it can be trained and applied to test data, Section 3 presents experimental results on the TIMIT speech corpus and concluding remarks and directions for future work are given in Section 4.”
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System (2016, methodology): “…speech signal to graphemes [13], we propose a simple(r) architecture (23 millions of parameters for our best model, vs. 100 millions of parameters in [1]) based on convolutional networks for the acoustic model, toppled with a graph transformer network [4], trained with a simpler sequence criterion.”
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism (2016, background): “Some of the earlier successes in this task have, for instance, been handwriting recognition (Bottou et al., 1997; Graves et al., 2009) and speech recognition (Graves et al., 2006; Chorowski et al., 2015).”
Fully Connected Deep Structured Networks (2015, background): “d in work by Simonyan and Zisserman [31]. Ideas along the lines of joint training were discussed within machine learning and computer vision as early as the 90’s in work done by Bridle [2] and Bottou [1]. More recently [5, 27, 22, 6, 28, 25] incorporate non-linearities into unary potentials but gene”
UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory (2016, background): “asing performance in the remaining tasks (higher is better for all tasks). synergy between them - this has been a recurring theme in computer vision, e.g. for integrating segmentation and recognition [9, 47, 50, 53, 68, 72, 99], and we believe that successfully addressing this it is imperative to ha”
Learning Deep Structured Models (2014, methodology): “For document processing systems, Bottou et al. (1997) propose Graph Transformer Networks to jointly optimize sub-tasks, such as word segmentation and character recognition.”
Globally Normalized Transition-Based Neural Networks (2016, methodology): “…but perform beam search for maintaining multiple hypotheses and introduce global normalization with a conditional random ﬁeld (CRF) ob-jective (Bottou et al., 1997; Le Cun et al., 1998; Lafferty et al., 2001; Collobert et al., 2011) to overcome the label bias problem that locally normalized…”
Structural-RNN: Deep Learning on Spatio-Temporal Graphs (2015, background): “They have been applied to multiple problems including image segmentation, pose estimation, document processing [64, 6, 38, 3] etc.”
Estimating phoneme class conditional probabilities from raw speech signal using convolutional neural networks (2013, methodology): “Also, in this work the CRF is optimized independently from the CNN, but joint training of the two models is in fact possible [23], and might lead to better performances.”
Object Recognition with Gradient-Based Learning (1999, background): “Networks of graph-manipulating modules are called Graph Transformer Networks [ 3 , 19].”
Matrix Backpropagation for Deep Networks with Structured Layers (2015, methodology): “Our architecture is also complementary to structured output formulations such as MRFs[5, 33, 8] which have been demonstrated to provide useful smoothing on top of highperforming CNN pixel classifier predictions [30].”
Differentiable Weighted Finite-State Transducers (2020, background): “Related to this work, and inspiring the name of our framework, are the graph transformer networks of Bottou et al. (1997).”