Managing GPU Concurrency in Heterogeneous Architectures

Onur Kayiran, Nachiappan Chidambaram Nachiappan, Adwait Jog, Rachata Ausavarungnirun, Mahmut T. Kandemir, Gabriel H. Loh, Onur Mutlu, Chita R. Das

Research output: Chapter in Book/Report/Conference proceedingConference contribution

59 Scopus citations


Heterogeneous architectures consisting of general-purpose CPUs and throughput-optimized GPUs are projected to be the dominant computing platforms for many classes of applications. The design of such systems is more complex than that of homogeneous architectures because maximizing resource utilization while minimizing shared resource interference between CPU and GPU applications is difficult. We show that GPU applications tend to monopolize the shared hardware resources, such as memory and network, because of their high thread-level parallelism (TLP), and discuss the limitations of existing GPU-based concurrency management techniques when employed in heterogeneous systems. To solve this problem, we propose an integrated concurrency management strategy that modulates the TLP in GPUs to control the performance of both CPU and GPU applications. This mechanism considers both GPU core state and system-wide memory and network congestion information to dynamically decide on the level of GPU concurrency to maximize system performance. We propose and evaluate two schemes: one (CM-CPU) for boosting CPU performance in the presence of GPU interference, the other (CM-BAL) for improving both CPU and GPU performance in a balanced manner and thus overall system performance. Our evaluations show that the first scheme improves average CPU performance by 24%, while reducing average GPU performance by 11%. The second scheme provides 7% average performance improvement for both CPU and GPU applications. We also show that our solution allows the user to control performance trade-offs between CPUs and GPUs.

Original languageEnglish (US)
Title of host publicationProceedings - 47th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2014
PublisherIEEE Computer Society
Number of pages13
ISBN (Electronic)9781479969982
StatePublished - Jan 15 2015
Event47th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2014 - Cambridge, United Kingdom
Duration: Dec 13 2014Dec 17 2014

Publication series

NameProceedings of the Annual International Symposium on Microarchitecture, MICRO
ISSN (Print)1072-4451


Other47th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2014
Country/TerritoryUnited Kingdom

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture


Dive into the research topics of 'Managing GPU Concurrency in Heterogeneous Architectures'. Together they form a unique fingerprint.

Cite this