Deep Learning

By Yann LeCun, Yoshua Bengio, Geoffrey E. Hinton. 2015

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

Citations in the corpus

Neural Network Acceptability Judgments (2018, methodology): “ting the Black Box Recurrent neural network models like the Long Short-Term Memory (LSTM) networks we use (Hochreiter and Schmidhuber, 1997) can discover some forms of structure in raw language data (LeCun et al., 2015). These models are widely used to encode features of sentences in ﬁxed-length sen”
The Curious Case of Neural Text Degeneration (2019, background): “One common approach is to shape the distribution through temperature (Goodfellow et al., 2016; Ficler and Goldberg, 2017; Fan et al., 2018).”
What makes ImageNet good for transfer learning? (2016, methodology): “Using ImageNet pre-trained CNN features, impressive results have been obtained on several image classification datasets [13, 35], object detection [16, 39], action recognition [41], human pose estimation [9], image segmentation [10], optical flow [47], image captioning [12, 22] and others [26].”
Scaling description of generalization with number of parameters in deep learning (2019, background): “Deep neural networks are very successful at various tasks including image classiﬁcation [25, 27] and speech recognition [19].”
High-dimensional dynamics of generalization error in neural networks (2017, background): “e behavior observed in simulation. Keywords: Neural networks, Generalization error, Random matrix theory 1. Introduction Deep learning approaches have attained high performance in a variety of tasks (LeCun, Bengio, and Hinton, 2015; Schmidhuber, 2015), yet their learning behavior remains opaque. . B”
A Survey of Machine Learning for Big Code and Naturalness (2017, background): “ame adaptations without external information (e.g. tests). Clones may indicate refactoring opportunities (that allow reusing the cloned code). White etal.[187]useautoencodersandrecurrentneuralnetworks[72]toﬁndclonesascodesnippetsthat share similar distributed representations. Using distributed vecto”
A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, background): “A recent line of work has argued that representations that are disentangled are an important step towards a better representation learning (Bengio et al., 2013; Peters et al., 2017; LeCun et al., 2015; Bengio et al., 2007; Schmidhuber, 1992; Lake et al., 2017; Tschannen et al., 2018).”
End-to-End Training of Deep Visuomotor Policies (2015, methodology): “ ways that dier from the training. The success of CNNs on exceedingly challenging vision tasks suggests that this class of models is capable of learning invariance to irrelevant distractor features (LeCun et al., 2015), and in principle this issue can be addressed by training the policy in a variety”
Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, background): “…model (CGM) formalism (Pearl, 2009; Peters et al., 2017), which operates on semantically meaningful high-level variables, with deep neural networks (Goodfellow et al., 2016), which excel on unstructured, low-level, high-dimensional © 2022 S. Lachapelle, P. Rodrı́guez López, Y. Sharma, K. Everett,…”
Gradient-Based Neural DAG Learning (2019, background): “Our first contribution is to extend the work of [18] to deal with nonlinear relationships between variables using neural networks (NN) [4].”
The jamming transition as a paradigm to understand the loss landscape of deep neural networks (2018, background): “[2] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton,”
Why Should We Add Early Exits to Neural Networks? (2020, background): “…that f is composed by the cascade of L differentiable operators f 1 , . . . , f L : where ◦ denotes function composition such that Examples of operations that can act as components in (1) include convolutional layers for images [ 21 ] , batch normalization, self-attention [ 71 ] , and many others.”
NIPS 2016 Tutorial: Generative Adversarial Networks (2016, methodology): “For more information about the chain rule of probability used to define FVBNs or about the effect of deterministic transformations on probability densities as used to define nonlinear ICA models, see chapter 3 of Goodfellow et al. (2016).”
Towards Better Decoding and Language Model Integration in Sequence to Sequence Models (2016, background): “Deep learning [1] has led to many breakthroughs including speech and image recognition [2, 3, 4, 5, 6, 7].”
Iterative Alternating Neural Attention for Machine Reading (2016, methodology): “The sequence X is processed using a recurrent neural network encoder (Goodfellow et al., 2016) with gated recurrent units (GRU) (Cho et al.”