GPUs and the Future of Parallel Computing

By S. Keckler, William (Bill) J. Dally, Brucek Khailany, Michael Garland, D. Glasco. IEEE Micro, 2011

650 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Hardware and Architecture

View Paper: https://www.semanticscholar.org/paper/afce4d0ba3ab134bdbfb35fd23e17de358acb9a8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Hardware and Architecture

Citations in the corpus

Memory devices and applications for in-memory computing (2020, neutral): “The current approaches, such as the use of hundreds of processors in parallel (for example, graphics processing units 3 ) or application-specific processors 4,5 that are custom designed for specific applications, are not likely to fully overcome the challenge of data movement.”
NDA: Near-DRAM acceleration architecture leveraging commodity DRAM devices and standard memory modules (2015, methodology): “For example, transferring data from off-chip DRAM devices through the cache hierarchy to the register file consumes about two orders of magnitude more energy than performing a floating-point operation [1].”
Pinatubo: A processing-in-memory architecture for bulk bitwise operations in emerging non-volatile memories (2016, background): “Moving data is much more expensive than computing itself: a DRAM access consumes 200 times more energy than a ﬂoating-point operation [14].”
Large-Scale Optical Neural Networks based on Photoelectric Multiplication (2018, methodology): “Transimpedance amplifiers designed for optical interconnects operate at the ∼ 100 fJ range [59, 26], while ADCs in the fewpJ/sample regime are available [60] and simple arithmetic (for the activation function) can be performed at the pJ scale [15, 16, 17].”
Gunrock: a high-performance graph processing library on the GPU (2015, background): “Though the GPU’s excellent peak throughput and energy efﬁciency [16] have been demonstrated across many application domains, these applications often exploit regular, structured parallelism.”
Attojoule Optoelectronics for Low-Energy Information Processing and Communications (2016, methodology): “inside electronic machines is used to communicate; for example, even on silicon chips 50–80% of gates are for the “repeater” amplifiers in long interconnect lines on the chip [7], and information also has to be driven off chip [5], and over data links and networks [8], [9], at much greater energies ”
TOP-PIM: throughput-oriented programmable processing in memory (2014, methodology): “For both host and PIM, we assume 2pJ/b for DRAM access itself [27, 29].”
PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory (2016, neutral): “For example, the data transfer between CPUs and off-chip memory consumes two orders of magnitude more energy than a floating point operation [1].”
OWL: cooperative thread array aware scheduling techniques for improving GPGPU performance (2013, background): “00 range of applications due to their immense computing power compared to CPUs [1, 6, 7, 25, 28, 48].”
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review (2019, background): “[73] proved that the highest energy expense results from accessing the off-chip DRAM memory for data movement rather than computation.”
Google Workloads for Consumer Devices: Mitigating Data Movement Bottlenecks (2018, background): “This is due to the fact that the energy cost of moving data is orders of magnitude higher than the energy cost of computation [80].”
ad-heap: an Efficient Heap Data Structure for Asymmetric Multicore Processors (2014, background): “On the other hand, the delete-max operation reconstructs the heap property in a top-down order that does not have any data parallelism either, but executes multiple (logd n in the worst case) lower-level parallelizable find-maxchild operations.”