Pushing the Performance Envelope of DNN-based Recommendation Systems Inference on GPUs

Rishabh Jain, Vivek M. Bhasi, Adwait Jog, Anand Sivasubramaniam, Mahmut Kandemir, Chitaranjan Das

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Personalized recommendation is a ubiquitous appli-cation on the internet, with many industries and hyperscalers extensively leveraging Deep Learning Recommendation Models (DLRMs) for their personalization needs (like ad serving or movie suggestions). With growing model and dataset sizes pushing computation and memory requirements, GPUs are being increasingly preferred for executing DLRM inference. However, serving newer DLRMs, while meeting acceptable latencies, continues to remain challenging, making traditional deployments increasingly more GPU-hungry, resulting in higher inference serving costs. In this paper, we show that the embedding stage continues to be the primary bottleneck in the GPU inference pipeline, leading up to a 3.2 x embedding-only performance slowdown. To thoroughly grasp the problem, we conduct a detailed microarchitecture characterization and highlight the presence of low occupancy in the standard embedding kernels. By leveraging direct compiler optimizations, we achieve optimal occupancy, pushing the performance by up to 53 %. Yet, long memory latency stalls continue to exist. To tackle this challenge, we propose spe-cialized plug-And-play-based software prefetching and L2 pinning techniques, which help in hiding and decreasing the latencies. Further, we propose combining them, as they complement each other. Experimental evaluations using AI00 GPUs with large models and datasets show that our proposed techniques improve performance by up to 103% for the embedding stage, and up to 77 % for the overall D LRM inference pipeline.

Original languageEnglish (US)
Title of host publicationProceedings - 2024 57th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2024
PublisherIEEE Computer Society
Pages1217-1232
Number of pages16
ISBN (Electronic)9798350350579
DOIs
StatePublished - 2024
Event57th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2024 - Austin, United States
Duration: Nov 2 2024Nov 6 2024

Publication series

NameProceedings of the Annual International Symposium on Microarchitecture, MICRO
ISSN (Print)1072-4451

Conference

Conference57th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2024
Country/TerritoryUnited States
CityAustin
Period11/2/2411/6/24

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Pushing the Performance Envelope of DNN-based Recommendation Systems Inference on GPUs'. Together they form a unique fingerprint.

Cite this