Anatomy of GPU memory system for multi-application execution

As GPUS make headway in the computing landscape span- ning mobile platforms, supercomputers, cloud and virtual desktop platforms, supporting concurrent execution of mul- Tiple applications in GPUS becomes essential for unlock- ing their full potential. However, unlike CPUs, multi- Application execution in GPUS is little explored. In this paper, we study the memory system of GPUS in a con- currently executing multi-application environment. We first present an analytical performance model for many-threaded architectures and show that the common use of misses-per- kilo-instruction (MPKI) as a proxy for performance is not accurate without considering the bandwidth usage of ap- plications. We characterize the memory interference of ap- plications and discuss the limitations of existing memory schedulers in mitigating this interference. We extend the analytical model to multiple applications and identify the key metrics to control various performance metrics. We conduct extensive simulations using an enhanced version of GPGPU-Sim targeted for concurrently executing multiple applications, and show that memory scheduling decisions based on MPKI and bandwidth information are more effec- Tive in enhancing throughput compared to the traditional FR-FCFS and the recently proposed RR FR-FCFS policies.