Large-Scale Learning of Embeddings with Reconstruction Sampling

By Yann Dauphin, Xavier Glorot, Yoshua Bengio. 2011

42 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Pylearn2: a machine learning research library (2013, methodology): “Pylearn2 has implementations of several models that were de v loped at LISA, including denoising auto-encoders (DAEs) [53], contractive auto-encoders (CAEs) [42] including higher-order CAEs [43], spike-and-slab RBMs (ssRBMs) [11] including ssR BMs with pooling [12], reconstruction sampling autoencod”
Representation Learning: A Review and New Perspectives (2012, methodology): “Finally, it has also been successfully used to surpass the state-of-the-art in sentiment analysis (Glorot et al., 2011b; Socher et al., 2011b).”
Marginalized Denoising Autoencoders for Domain Adaptation (2012, methodology): “We combine the concept of “pivot features” from Blitzer et al. (2006) and the use of most-frequent features from Glorot et al. (2011).”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, methodology): “In Glorot et al. (2011a) it was shown that an asymmetric but hard-limiting non-linearity such as the rectifier (s(a) = max(0, a), see also (Nair and Hinton, 2010)) actually worked very well (but should not be used for output units), in spite of the prior belief that the fact that when hidden units…”
An Autoencoder Approach to Learning Bilingual Word Representations (2014, methodology): “We note that, additionally, we could have used the stochastic approach proposed by Dauphin et al. (2011) for re-constructing binary bag-of-words representations of documents, to further improve the efﬁciency of training.”
Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives (2012, background): “…al. , 2011a; Nair and Hinton, 2010) showing that the choice of hidden units non-linearity could inﬂuence both training and generalization performance, with particularly interesting results obtained with sparse rectifying units (Jarrett et al. , 2009; Glorot et al. , 2011a; Nair and Hinton, 2010).”
Clustering is Efficient for Approximate Maximum Inner Product Search (2015, methodology): “outlook of approximate MIPS is akin to techniques that each time heuristically sample a smaller candidate subset of the output nodes to evaluate, as e.g. in the biased importance sampling approach of [9, 10], or when employing Noise Contrastive Estimation [11] as proposed in [12]. But while these te”
Stochastic Ratio Matching of RBMs for Sparse High-Dimensional Inputs (2013, methodology): “Note that Dauphin et al. (2011) relies on the classification error because there is no accepted performance measure for DAEs.”
A Neural Autoregressive Topic Model (2012, background): “[12] have trained neural network autoencoders on documents in their binary bag of words representation, but such neural networks are not generative models of documents.”
Text summarization using unsupervised deep learning (2017, unknown): “Genest et al. (2011) and Glorot et al. (2011) presented a re-sampling algorithm to solve this problem for AE.”