Distributed Asynchronous Online Learning for Natural Language Processing

By Kevin Gimpel, Dipanjan Das, Noah A. Smith. 2010

44 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Optimal Distributed Online Prediction Using Mini-Batches (2010, neutral): “The idea of using mini-batches in stochastic and online learning is not new, and has been previously explored in both the serial and parallel settings (see, e.g., Shalev-Shwartz et al., 2007; Gimpel et al., 2010).”
Automatic differentiation in machine learning: a survey (2015, background): “Improvements in training time can be realized by using online or distributed training algorithms (Gimpel et al, 2010).”
A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order (2016, methodology): “2 Asynchronous Stochastic Gradient Descent (ASGD) ASGD has been widely used to solve deep learning [7, 26, 36], NLP [4, 13], and many other important machine learning problems [25].”
A Survey of Deep Learning for Mathematical Reasoning (2022, unknown): “Existing work about online learning (LeCun et al., 1998; Gimpel et al., 2010; Liang and Klein, 2009) and incorporating humans”
Proposal Scaling Distributed Machine Learning with System and Algorithm Co-design (2016, unknown): “For example, in [58], asynchronous communication is suggested.”
Fully Distributed Privacy Preserving Mini-batch Gradient Descent Learning (2015, unknown): “This allows for fast distributed implementations as well [11].”
Efficient mini-batch training for stochastic optimization (2014, methodology): “[11] proposed to use asynchronous communication.”
Hope and Fear for Discriminative Training of Statistical Translation Models (2012, background): “The more communication-intensive option, known asasynchronousSGD, is to maintain a single weight vector and forp parallel learners to update it simultaneously (Langford et al., 2009; Gimpel et al., 2010).”
Optimal Distributed Online Prediction (2011, methodology): “As the name distributed mini-batch suggests, our approach makes use of the idea of mini-batches, which is not new and has been previously explored in both the serial and parallel setting (see, e.g., Delalleau & Bengio, 2007; Gimpel et al., 2010).”