Compete to Compute

By Rupesh Kumar Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino J. Gomez, J. Urgen Schmidhuber. 2013

225 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Training Very Deep Networks (2015, background): “This enables the study of extremely deep and efficient architectures.”
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015, background): “, increased depth [29, 33], enlarged width [37, 28], and the use of smaller strides [37, 28, 2, 29]), new nonlinear activations [24, 23, 38, 22, 31, 10], and sophisticated layer designs [33, 12].”
Instance-Aware Semantic Segmentation via Multi-task Network Cascades (2015, background): “NMS is similar to max pooling, maxout [11], or other local competing layers [28], which are implemented as routers of forward/backward pathways.”
Striving for Simplicity: The All Convolutional Net (2014, background): “Among these the most notable directions are work on using more complex activation functions (Goodfellow et al., 2013; Lin et al., 2014; Srivastava et al., 2013) techniques for improving class inference (Stollenga et al., 2014; Srivastava & Salakhutdinov, 2013) as well as procedures for improved…”
Continual Learning with Deep Generative Replay (2017, methodology): “The most common experimental formulation used in continual learning literature [34, 18] is a simple image classification problem where the inputs are images from MNIST handwritten digit database [19], but pixel values of inputs are shuffled by a random permutation sequence unique to each task.”
An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks (2013, background): “Rectiﬁed linear (Jarrett et al., 2009; Glorot et al., 2011a): et al., 2013): Here g is a gating function. z is divided into disjoint blocks of size k , and g ( i, z ) is 1 if z i is the maximal element of its group.”
Meta Networks (2017, methodology): “Following the previous work on catastrophic forgetting in neural networks (Srivastava et al., 2013; Goodfellow et al., 2014; Kirkpatrick et al., 2016), we formulated two problems in a sequential manner.”
Deep Learning: Methods and Applications (2014, background): “[350] propose yet another new type of nonlinear units, called winner-take-all units.”
Learning Rapid-Temporal Adaptations (2017, background): “Systems built in such a train-and-then-test manner do not scale to complex, realistic environments and tasks: They require gluts of single task data and are prone to issues related to distributional shifts, such as catastrophic forgetting [37, 8, 16] and adversarial data points [39].”
Overcoming catastrophic forgetting with hard attention to the task (2018, methodology): “Next, we consider the permuted MNIST sequence of tasks (Srivastava et al., 2013).”
Deep Generative Dual Memory Network for Continual Learning (2017, background): “More recent works have explored activations like dropout (Goodfellow et al., 2015) and local winner-takes-all (Srivastava et al., 2013) to create sparse, less correlated feature representations.”
CORe50: a New Dataset and Benchmark for Continuous Object Recognition (2017, methodology): “For example, EWC classiﬁcation tests have been carried out on permuted MNIST task [9] [26], where the new tasks to learn are obtained by scrambling the pixel positions in the MNIST dataset.”
Less-forgetting Learning in Deep Neural Networks (2016, methodology): “…of Science and Technology, Daejeon, Republic of Korea, e-mail: { heechul, alswn0925, junmo.kim } @kaist.ac.kr. Jungwoo Ju is with the Division of Future Vehicle, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea, e-mail: veryju@kaist.ac.kr catastrophic forgetting [8].”
Overcoming Catastrophic Forgetting by Incremental Moment Matching (2017, methodology): “The first experiment is the disjoint MNIST experiment [4].”
Continual Learning with Adaptive Weights (CLAW) (2019, methodology): “A continual learner must be able to learn a new task, crucially, without forgetting previous tasks (Ring, 1995; Srivastava et al., 2013; Schwarz et al., 2018; Serra et al., 2018; Hu et al., 2019).”
Orthogonal Gradient Descent for Continual Learning (2019, background): “, 2006; Simonyan and Zisserman, 2014) this problem has been revived (Srivastava et al., 2013; Goodfellow et al., 2013) with many follow-up studies (Parisi et al.”