Pipelined BackPropagation for Context-Dependent Deep Neural Networks

By Xie Chen, Adam Eversole, Gang Li, Dong Yu, Frank Torsten Bernd Seide. 2012

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Khe Chai Sim
2. Mark John Francis Gales
3. Chunyang Wu
4. Panagiota Karanasou
5. Tal Ben-Nun
6. Torsten Hoefler
7. Andrei Ivanov
8. Nikoli Dryden

Citations in the corpus

• Deep Learning: Methods and Applications (2014, methodology): “Recently, SGD is shown to be effective for parallelizing over many machines with an asynchronous mode [69] or over multiple GPUs through pipelined BP [49].”
• Stimulated Deep Neural Network for Speech Recognition (2016, background): “In recent years, deep neural networks [1, 2, 3] (DNNs) have successfully been applied to acoustic models of state-of-the-art speech recognition systems.”
• Data Movement Is All You Need: A Case Study on Optimizing Transformers (2020, methodology): “Other optimizations, including model parallelism [76], [85], [101]–[105], pipeline parallelism [106]–[109], microbatching [110], and recomputation for memory reduction [111], [112] are all also applicable.”