Mixing floating- and fixed-point formats for neural network learning on neuroprocessors
By Davide Anguita, Benedict A. Gomes. 1996
Abstract We examine the efficient implementation of back-propagation (BP) type algorithms on TO [3], a vector processor with a fixed-point engine, designed for neural network simulation. Using Matrix Back Propagation (MBP) [2]we achieve an asymptotically optimal performance on TO (about 0.8 GOPS) for both forward and backward phases, which is not possible with the standard on-line BP algorithm. We use a mixture of fixed- and floating-point operations in order to guarantee both high efficiency and fast convergence. Though the most expensive computations are implemented in fixed-point, we achieve a rate of convergence that is comparable to the floating-point version. The time taken for conversion between fixed- and floating-point is also shown to be reasonably low.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Luca Oneto
- Jorge Luis Reyes-Ortiz
- Davide Anguita
- Xavier Parra
- J. Urgen Schmidhuber
- Alessandro Ghio
- David R Ha
- Vivek Sharma
Citations in the corpus
- Deep learning in neural networks: An overview (2014, background): “, 1997; Urlbe, 1999), and at exploiting standard hardware (e.g., Anguita et al., 1994; Muller et al., 1995; Anguita and Gomes, 1996), the new millennium brought a DL breakthrough in form of cheap, multiprocessor graphics cards or GPUs.”
- Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine (2012, background): “In ML, fixed-point arithmetic models have been previously studied [11, 12] initially because devices with floating-point units were unavailable or expensive.”