Distributed Deep Learning for Question Answering
By Minwei Feng, Bing Xiang, Bowen Zhou. 2015
8 citations recorded by the source corpus.
This paper is an empirical study of the distributed deep learning for question answering subtasks: answer selection and question classification. Comparison studies of SGD, MSGD, ADADELTA, ADAGRAD, ADAM/ADAMAX, RMSPROP, DOWNPOUR and EASGD/EAMSGD algorithms have been presented. Experimental results show that the distributed framework based on the message passing interface can accelerate the convergence speed at a sublinear scale. This paper demonstrates the importance of distributed training. For example, with 48 workers, a 24x speedup is achievable for the answer selection task and running time is decreased from 138.2 hours to 5.81 hours, which will increase the productivity significantly.
Subfields: Artificial Intelligence, Information Systems
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Information Systems
Citations in the corpus
- Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent (2017, methodology): “For image classification we train ResNet [He et al., 2015] with different number of layers on CIFAR-10 [Krizhevsky, 2009]; for speech recognition, we train both proprietary and public dataset on a proprietary CNN model that we get from our industry partner [Feng et al., 2016, Lin et al., 2017].”
- GaDei: On Scale-Up Training as a Service for Deep Learning (2016, unknown): “nts, which leads to a slower convergence rate[19, 21, 53, 58]. We explain why training with a small mini-batch size can effectively reduce staleness in Section 2.3. Additionally, several recent works [24, 39, 57, 58] demonstrate that ASGD can converge to a similar model accuracy ( 1%) as SGD after t”