Exploring the Hidden Dimension in Accelerating Convolutional Neural Networks

11 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation (2021, methodology): “One regime is op-level model parallelism [17, 18, 19, 20, 21, 22, 23], which partitions its involved tensors of each operator (layer), as shown in Figure 4(b).”
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization (2019, background): “input generation time. As neural network operations consist of dense numerical kernels such as matrix multiplication, these runtimes are low variance and largely independent of the speciﬁc input data [30, 52]. However, forward pass time per batch item decreases with increasing batch size due to impr”
torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models (2020, background): “necessary. Obviously, this requires certain part of the model be computed twice and overall training time would be increased. It is beneﬁtting to combine different types of parallelization strategies [16,14,26,12,9,11,7], and recent lines of research questions how to ﬁnd an optimal strategy 1 arXiv:”