More iteration space tiling
By M. Wolfe. 1989
Subdividing the iteration space of a loop into blocks or tiles with a fixed maximum size has several advantages. Tiles become a natural candidate as the unit of work for parallel task scheduling. Synchronization between processors can be done between tiles, reducing synchronization frequency (at some loss of potential parallelism). The shape and size of a tile can be optimized to take advantage of memory locality for memory hierarchy utilization. Vectorization and register locality naturally fits into the optimization within a tile, while parallelization and cache locality fits into optimization between tiles.
Subfields: Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Optimization & Training
- HyoukJoong Lee
- Ashish Vaswani
- Blake A. Hechtman
- Michael E. Wolf
- Noam Shazeer
- Peter G. Hawkins
- Mingsheng Hong
- Penporn Koanantakool
Citations in the corpus
- Mesh-TensorFlow: Deep Learning for Supercomputers (2018, methodology): “For example, a 512-core TPU cluster with a 16x16x2 toroidal network interconnect could be represented by a 3-dimensional mesh with shape [16, 16, 2], a two-dimensional mesh with shape [32, 16], a one-dimensional mesh with shape [512], etc.”
- A data locality optimizing algorithm (1991, background): “Blocking [9] or tiling [ 18 ] is a well-known technique that improves the data locality of numerical algorithms [1, 6, 7, 12, 13].”