Glow: Graph Lowering Compiler Techniques for Neural Networks

By Nadav Rotem, J. Fix, Saleem Abdulrasool, Summer Deng, Roman Dzhabarov, James Hegeman, Roman Levenstein, Bertrand A. Maher, Nadathur Rajagopalan Satish, J. Olesen and 3 more. 2018

337 citations recorded by the source corpus.

This paper presents the design of Glow, a machine learning compiler for heterogeneous hardware. It is a pragmatic approach to compilation that enables the generation of highly optimized code for multiple targets. Glow lowers the traditional neural network dataflow graph into a two-phase strongly-typed intermediate representation. The high-level intermediate representation allows the optimizer to perform domain-specific optimizations. The lower-level instruction-based address-only intermediate representation allows the compiler to perform memory-related optimizations, such as instruction scheduling, static memory allocation and copy elimination. At the lowest level, the optimizer performs machine-specific code generation to take advantage of specialized hardware features. Glow features a lowering phase which enables the compiler to support a high number of input operators as well as a large number of hardware targets by eliminating the need to implement all operators on all targets. The lowering phase is designed to reduce the input space and allow new hardware backends to focus on a small number of linear algebra primitives.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Hardware and Architecture, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Hardware and Architecture

In ML Systems & Hardware

Citations in the corpus

GSPMD: General and Scalable Parallelization for ML Computation Graphs (2021, background): “, ONNX [3], TVM Relay [29], Glow IR [30]).”
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (2020, methodology): “Although we developed the infrastructure in XLA, the techniques we describe here can be applied to intermediate representations in other machine learning frameworks (e.g., ONNX onn (2019), TVM Relay Roesch et al. (2018), Glow IR Rotem et al. (2018)).”
MLIR: Scaling Compiler Infrastructure for Domain Specific Computation (2021, neutral): “More narrowly construed by their application to machine learning, XLA [24], Glow [48] and TVM [49], address similar heterogeneous compilation objectives.”
Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning (2020, methodology): “For example, DL compilers [6, 14, 17, 30, 37] have been proposed to generate optimized codes for target hardware.”
Data Movement Is All You Need: A Case Study on Optimizing Transformers (2020, background): “Optimizing frameworks can also be applied to transformers (Google, 2020; Frostig et al., 2018; Bradbury et al., 2018; PyTorch Team, 2020; Rotem et al., 2018; Jia et al., 2019d;b;a; Chen et al., 2018; Paul G. Allen School of Computer Science & Engineering, University of Washington et al., 2017;…”
Gradient Compression Supercharged High-Performance Data Parallel DNN Training (2021, background): “However, in recent years, the fast-growing computing capability, driven by the booming of GPU architecture innovations [49] and domain-specific compiler techniques [14, 17, 57, 58], tends to result in more frequent and heavier gradient synchronization during data parallel DNN training.”
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better (2021, methodology): “The alternatives for XLA in the PyTorch world seem to be the Glow [115] and TensorComprehension [137] compilers.”
The Deep Learning Compiler: A Comprehensive Survey (2020, methodology): “The low-level IR in Glow is an instruction-based expression that operates on tensors referenced by addresses [27].”
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications (2018, background): “Automatic generation of fused kernels is an active area of research and early productization efforts are underway [10, 42, 57, 67].”
Dynamic Tensor Rematerialization (2020, background): “In other frameworks that feature a compilation step [31, 32, 33], it may be possible to eliminate much of this overhead by generating these structures in a compiler pass.”
Machine Learning at the Network Edge: A Survey (2019, methodology): “With increase in the dedicated hardware for ML, an important direction of future work is the development of compilers, such as Glow [124], that optimize neural network graphs for heterogeneous hardware.”
TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems (2020, methodology): “Graph Lowering (GLOW) (Rotem et al., 2018) TensorFlow-Native was an experimental Google system that compiled TensorFlow graphs into C++ code.”
MLIR: A Compiler Infrastructure for the End of Moore's Law (2020, neutral): “More narrowly construed by their application to machine learning, XLA [57], Glow [40] and TVM [11], address similar heterogeneous compilation objectives.”
Machine Learning at Facebook: Understanding Inference at the Edge (2019, methodology): “First option is to compile applications containing ML models to platform-speciﬁc object code using, for example, Glow [10], XLA [11], or TVM [12].”
Benchmarking TPU, GPU, and CPU Platforms for Deep Learning (2019, methodology): “Software/- compiler approaches include better operation fusion [1, 11, 44], more aggressive data quantization [6], and weights and gradients compression [17, 38].”