Relay: a new IR for machine learning frameworks

By Jared Roesch, Steven Lyubomirsky, Logan Weber, Josh Pollock, Marisa Kirisame, Tianqi Chen, Zachary Tatlock. 2018

122 citations recorded by the source corpus.

Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they be efficient, expressive, and portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to better accommodate them we propose a new high-level intermediate representation (IR) called Relay. Relay is being designed as a purely-functional, statically-typed language with the goal of balancing efficient compilation, expressiveness, and portability. We discuss the goals of Relay and highlight its important design constraints. Our prototype is part of the open source NNVM compiler framework, which powers Amazon's deep learning framework MxNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Hardware and Architecture, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Quoc V. Le
2. Orhan Firat
3. Yonghui Wu
4. Graham Neubig
5. Yu Zhang
6. Yanping Huang
7. Ruoming Pang
8. Ankur Bapna

In Computer Vision and Pattern Recognition

1. Quoc V. Le
2. Orhan Firat
3. Luc Van Gool
4. Yonghui Wu
5. Dacheng Tao
6. Yanping Huang
7. Yi Yang
8. Trevor Darrell

In Hardware and Architecture

1. Yanping Huang
2. Z. Chen
3. Dmitry Lepikhin
4. Noam Shazeer
5. Maxim Krikun
6. Shibo Wang
7. Yuanzhong Xu
8. Orhan Firat

In Model Architectures

1. Orhan Firat
2. Yonghui Wu
3. Yanping Huang
4. Ruoming Pang
5. Z. Chen
6. Maxim Krikun
7. Yuanzhong Xu
8. Dmitry Lepikhin

Citations in the corpus

• GSPMD: General and Scalable Parallelization for ML Computation Graphs (2021, background): “, ONNX [3], TVM Relay [29], Glow IR [30]).”
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (2020, methodology): “Although we developed the infrastructure in XLA, the techniques we describe here can be applied to intermediate representations in other machine learning frameworks (e.g., ONNX onn (2019), TVM Relay Roesch et al. (2018), Glow IR Rotem et al. (2018)).”
• Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement (2020, methodology): “More specifically, our TVM-based on-device system first converts a TensorFlow graph to a Relay graph [38] and complies the code to Firefly-RK3399 using LLVM.”