Efficient methods and hardware for deep learning

By B. Dally. 2017

65 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

I-BERT: Integer-only BERT Quantization (2021, neutral): “These techniques can be broadly categorized into (1) model pruning [18, 31, 35, 38, 40, 67], (2) knowledge distillation [21, 39, 43, 49, 70], (3) efficient neural architecture design [23, 24, 37, 51, 57], (4) hardware and neural architecture co-design [16, 17, 22, 29, 64], and (5) quantization [5, 7”
HAWQV3: Dyadic Neural Network Quantization (2020, unknown): “…(Iandola et al., 2016; Sandler et al., 2018; Tan & Le, 2019); (ii) Co-designing NN architecture and hardware together (Gholami et al., 2018; Han & Dally, 2017; Howard et al., 2019; Wu et al., 2019); (iii) Pruning redundant ﬁlters (Han et al., 2015; LeCun et al., 1990; Li et al., 2016; Mao…”
HAQ: Hardware-Aware Automated Quantization (2018, methodology): “For MobileNets which are already very compactly designed, our framework can preserve the performance to some extent; while Deep Compression significantly degrades the performance especially when the model size is very small.”
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks (2019, background): “Furthermore, the continued slowdown of Moore’s Law and Dennard scaling (Han, 2017) suggests that there exists a point in time when we must compress our models and carefully evaluate our choice of the neural architecture.”