TVM: An Automated End-to-End Optimizing Compiler for Deep Learning

By Tianqi Chen, Thierry Moreau, Ziheng Jiang, Haichen Shen, Eddie Q. Yan, Yuwei Hu, Luis Ceze, Carlos Guestrin, Arvind Krishnamurthy. 2018

Subfields: Efficiency & Compression, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

In Optimization & Training

Citations in the corpus

Pathways: Asynchronous Distributed Dataflow for ML (2022, positive): “Wherever possible the frameworks submit graphs of compiled functions to a “just in time” (JIT) compiler (Chen et al., 2018; TensorFlow, 2019) that is able to exploit optimizations like layout assignment and fusion that can substantially improve the efﬁciency of the resulting accelerator code.”
GSPMD: General and Scalable Parallelization for ML Computation Graphs (2021, background): “Compilation time is an important usability concern because modern ML frameworks often include JIT optimizations and compilation, especially for those targeting custom accelerators [2, 9, 30], and parallelizing the compilation can be non-trivial because operators in different programsmay need to be g”
Longformer: The Long-Document Transformer (2020, methodology): “Using TVM, we describe our form of banded matrix multiplication in high-level python constructs, then TVM generates the corresponding CUDA code and compiles it for GPUs.”
TSM: Temporal Shift Module for Efficient Video Understanding (2018, methodology): “Thus the deployment of our framework is hardware friendly: we only need to support the operations in 2D CNNs, which are already welloptimized at both framework level (CuDNN [6], MKL-DNN, TVM [5]) and hardware level (CPU/GPU/TPU/FPGA).”
DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale (2022, methodology): “Several compilers and runtimes exist to facilitate the deployment of models, such as TVM [28], ONNXRuntime [29] and TensorRT [30].”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “[154] shows that INT8 inference of popular computer vision models, including ResNet50 [87], VGG-19 [216], and inceptionV3 [222] using TVM [32] quantization library, can achieve 3.89 , 3.32 , and 5.02 speedup on NVIDIA GTX 1080, respectively.”
Timeloop: A Systematic Approach to DNN Accelerator Evaluation (2019, background): “Finally, recent work on sophisticated compilers for DNN workloads [4], [31] have built-in cost models, though the objective of those models is simply to provide enough ﬁdelity to guide the optimizer rather than to provide accurate energy and performance projections for architecture design-space…”
Machine Learning at Facebook: Understanding Inference at the Edge (2019, methodology): “First option is to compile applications containing ML models to platform-speciﬁc object code using, for example, Glow [10], XLA [11], or TVM [12].”
Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement (2020, methodology): “In this work, we extend a deep learning compiler stack - TVM [37] to support the core temporal shift operation required for TS-CAN.”
Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning (2020, methodology): “Moreover, Nimble outperforms state-of-the-art inference systems, TensorRT [3] and TVM [14], by up to 2.”
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution (2019, methodology): “OctConv networks are compiled by TVM [5] v0.”
Machine Learning at the Network Edge: A Survey (2019, background): “TVM is a compiler that has the capability of optimizing code by searching and detecting optimized tensor operators [27].”
DistIR: An Intermediate Representation for Optimizing Distributed Neural Networks (2021, background): “TVM [6] is an end-to-end optimizing compiler for DNNs, but to our knowledge it does not consider distribution.”
Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization (2019, methodology): “, evolutionary and simulated annealing algorithms for compiler optimization [39], [40] and one-shot decision theory in operation research [41].”
Transformers: State-of-the-Art Natural Language Processing (2020, unknown): “, 2018) and TVM (Chen et al., 2018).”
On-Device Training Under 256KB Memory (2022, methodology): “Edge deep learning inference frameworks like TVM [17], TF-Lite [3], NCNN [2], etc . provide a slim runtime, but lack the support for back-propagation.”