Implementation of a Practical Distributed Calculation System with Browsers and JavaScript, and Application to Distributed Deep Learning
By Ken Miura, Tatsuya Harada. 2015
10 citations recorded by the source corpus.
Deep learning can achieve outstanding results in various fields. However, it requires so significant computational power that graphics processing units (GPUs) and/or numerous computers are often required for the practical application. We have developed a new distributed calculation framework called "Sashimi" that allows any computer to be used as a distribution node only by accessing a website. We have also developed a new JavaScript neural network framework called "Sukiyaki" that uses general purpose GPUs with web browsers. Sukiyaki performs 30 times faster than a conventional JavaScript library for deep convolutional neural networks (deep CNNs) learning. The combination of Sashimi and Sukiyaki, as well as new distribution algorithms, demonstrates the distributed deep learning of deep CNNs only with web browsers on various devices. The libraries that comprise the proposed methods are available under MIT license at this http URL
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Cong Xie
- Oluwasanmi Koyejo
- Indranil Gupta
- Sai Praneeth Karimireddy
- Lie He
- Martin Jaggi
- Sebastian U. Stich
- Peter Richtárik
In Computer Vision and Pattern Recognition
- Oluwasanmi Koyejo
- Cong Xie
- Indranil Gupta
- Martin Jaggi
- Sai Praneeth Karimireddy
- Lie He
- Sebastian U. Stich
- Dan Alistarh
Citations in the corpus
- Zeno++: Robust Fully Asynchronous SGD (2019, background): “istributed settings has attracted increasing attention in the machine-learning community [1, 2, 3, 5, 13, 14, 15, 16]. In certain distributed settings,suchasfederatedlearning[7,8]andvolunteercomputing[10,11],theworkers/contributors can be unreliable. The computation nodes can be personal mobile devi”
- Zeno: Byzantine-suspicious stochastic gradient descent (2018, background): “In certain distributed settings, such as federated learning [9, 10] and volunteer computing [14, 16], the workers/contributors can be unreliable.”
- Zeno++: robust asynchronous SGD with arbitrary number of Byzantine workers (2019, background): “In certain distributed settings, such as federated learning [7, 8] and volunteer computing [10, 11], the workers/contributors can be unreliable.”
- Byzantine-Robust Learning on Heterogeneous Datasets via Resampling (2020, background): “Typical examples of this is distributed training of neural networks on public cloud, or volunteer computing Meeds et al. (2015); Miura and Harada (2015).”
- Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance (2018, background): “In volunteer computing (Meeds et al., 2015; Miura & Harada, 2015), it is reasonable for the coordinator to assign disjoint tasks/datasets to different workers.”