Orchestrated scheduling and prefetching for GPGPUs

Adwait Jog, Onur Kayiran, Asit K. Mishra, Mahmut T. Kandemir, Onur Mutlu, Ravishankar Iyer, Chita R. Das

Research output: Chapter in Book/Report/Conference proceedingConference contribution

142 Scopus citations


In this paper, we present techniques that coordinate the thread scheduling and prefetching decisions in a General Purpose Graphics Processing Unit (GPGPU) architecture to better tolerate long memory latencies. We demonstrate that existing warp scheduling policies in GPGPU architectures are unable to effectively incorporate data prefetching. The main reason is that they schedule consecutive warps, which are likely to access nearby cache blocks and thus prefetch accurately for one another, back-to-back in consecutive cycles. This either 1) causes prefetches to be generated by a warp too close to the time their corresponding addresses are actually demanded by another warp, or 2) requires sophisticated prefetcher designs to correctly predict the addresses required by a future "far-ahead" warp while executing the current warp. We propose a new prefetch-aware warp scheduling policy that overcomes these problems. The key idea is to separate in time the scheduling of consecutive warps such that they are not executed back-to-back. We show that this policy not only enables a simple prefetcher to be effective in tolerating memory latencies but also improves memory bank parallelism, even when prefetching is not employed. Experimental evaluations across a diverse set of applications on a 30-core simulated GPGPU platform demonstrate that the prefetch-aware warp scheduler provides 25% and 7% average performance improvement over baselines that employ prefetching in conjunction with, respectively, the commonly-employed round-robin scheduler or the recently-proposed two-level warp scheduler. Moreover, when prefetching is not employed, the prefetch-aware warp scheduler provides higher performance than both of these baseline schedulers as it better exploits memory bank parallelism.

Original languageEnglish (US)
Title of host publicationISCA 2013 - 40th Annual International Symposium on Computer Architecture, Conference Proceedings
Number of pages12
StatePublished - 2013
Event40th Annual International Symposium on Computer Architecture, ISCA 2013 - Tel-Aviv, Israel
Duration: Jun 23 2013Jun 27 2013

Publication series

NameProceedings - International Symposium on Computer Architecture
ISSN (Print)1063-6897


Other40th Annual International Symposium on Computer Architecture, ISCA 2013

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture


Dive into the research topics of 'Orchestrated scheduling and prefetching for GPGPUs'. Together they form a unique fingerprint.

Cite this