Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs

Krishnakumar Nair; Avinash Chandra Pandey; Siddappa Karabannavar; Meena Arunachalam; John Kalamatianos; Varun Agrawal; Saurabh Gupta; Ashish Sirasao; Elliott Delaye; Steve Reinhardt; Rajesh Vivekanandham; Ralph Wittig; Vinod Kathail; Padmini Gopalakrishnan; Satyaprakash Pareek; Rishabh Jain; Mahmut Taylan Kandemir; Jun Liang Lin; Gulsum Gudukbay Akbulut; Chita R. Das

doi:10.1109/MM.2024.3423785

Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs

Krishnakumar Nair, Avinash Chandra Pandey, Siddappa Karabannavar, Meena Arunachalam, John Kalamatianos, Varun Agrawal, Saurabh Gupta, Ashish Sirasao, Elliott Delaye, Steve Reinhardt, Rajesh Vivekanandham, Ralph Wittig, Vinod Kathail, Padmini Gopalakrishnan, Satyaprakash Pareek, Rishabh Jain, Mahmut Taylan Kandemir, Jun Liang Lin, Gulsum Gudukbay Akbulut, Chita R. Das

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

Deep learning recommendation models (DLRMs) are deployed extensively to support personalized recommendations and consume a large fraction of artificial intelligence (AI) cycles in modern datacenters with embedding stage being a critical component. Modern CPUs execute a lot of DLRM cycles because they are cost effective compared to GPUs and other accelerators. Our paper addresses key bottlenecks in accelerating the embedding stage on CPUs. Specifically, this work 1) explores novel threading schemes that parallelize embedding bag, 2) pushes the envelope on realized bandwidth by improving data reuse in caches, and 3) studies the impact of parallelization on load imbalance. The new embedding bag kernels have been prototyped in the ZenDNN software stack. When put together, our work on fourth generation EPYC processors achieve up to 9.9x improvement in embedding bag performance over state-of-the-art implementations, and improve realized bandwidth of up to 5.7x over DDR bandwidth.

Original language	English (US)
Pages (from-to)	44-51
Number of pages	8
Journal	IEEE Micro
Volume	44
Issue number	6
DOIs	https://doi.org/10.1109/MM.2024.3423785
State	Published - 2024

All Science Journal Classification (ASJC) codes

Software
Hardware and Architecture
Electrical and Electronic Engineering

Access to Document

10.1109/MM.2024.3423785

Cite this

Nair, K., Pandey, A. C., Karabannavar, S., Arunachalam, M., Kalamatianos, J., Agrawal, V., Gupta, S., Sirasao, A., Delaye, E., Reinhardt, S., Vivekanandham, R., Wittig, R., Kathail, V., Gopalakrishnan, P., Pareek, S., Jain, R., Kandemir, M. T., Lin, J. L., Akbulut, G. G., & Das, C. R. (2024). Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs. IEEE Micro, 44(6), 44-51. https://doi.org/10.1109/MM.2024.3423785

@article{d3547eb0aaed40c8a564e7ade41fcfb5,

title = "Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs",

abstract = "Deep learning recommendation models (DLRMs) are deployed extensively to support personalized recommendations and consume a large fraction of artificial intelligence (AI) cycles in modern datacenters with embedding stage being a critical component. Modern CPUs execute a lot of DLRM cycles because they are cost effective compared to GPUs and other accelerators. Our paper addresses key bottlenecks in accelerating the embedding stage on CPUs. Specifically, this work 1) explores novel threading schemes that parallelize embedding bag, 2) pushes the envelope on realized bandwidth by improving data reuse in caches, and 3) studies the impact of parallelization on load imbalance. The new embedding bag kernels have been prototyped in the ZenDNN software stack. When put together, our work on fourth generation EPYC processors achieve up to 9.9x improvement in embedding bag performance over state-of-the-art implementations, and improve realized bandwidth of up to 5.7x over DDR bandwidth.",

author = "Krishnakumar Nair and Pandey, {Avinash Chandra} and Siddappa Karabannavar and Meena Arunachalam and John Kalamatianos and Varun Agrawal and Saurabh Gupta and Ashish Sirasao and Elliott Delaye and Steve Reinhardt and Rajesh Vivekanandham and Ralph Wittig and Vinod Kathail and Padmini Gopalakrishnan and Satyaprakash Pareek and Rishabh Jain and Kandemir, {Mahmut Taylan} and Lin, {Jun Liang} and Akbulut, {Gulsum Gudukbay} and Das, {Chita R.}",

note = "Publisher Copyright: {\textcopyright} 1981-2012 IEEE.",

year = "2024",

doi = "10.1109/MM.2024.3423785",

language = "English (US)",

volume = "44",

pages = "44--51",

journal = "IEEE Micro",

issn = "0272-1732",

publisher = "IEEE Computer Society",

number = "6",

}

Nair, K, Pandey, AC, Karabannavar, S, Arunachalam, M, Kalamatianos, J, Agrawal, V, Gupta, S, Sirasao, A, Delaye, E, Reinhardt, S, Vivekanandham, R, Wittig, R, Kathail, V, Gopalakrishnan, P, Pareek, S, Jain, R, Kandemir, MT, Lin, JL, Akbulut, GG & Das, CR 2024, 'Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs', IEEE Micro, vol. 44, no. 6, pp. 44-51. https://doi.org/10.1109/MM.2024.3423785

TY - JOUR

T1 - Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs

AU - Nair, Krishnakumar

AU - Pandey, Avinash Chandra

AU - Karabannavar, Siddappa

AU - Arunachalam, Meena

AU - Kalamatianos, John

AU - Agrawal, Varun

AU - Gupta, Saurabh

AU - Sirasao, Ashish

AU - Delaye, Elliott

AU - Reinhardt, Steve

AU - Vivekanandham, Rajesh

AU - Wittig, Ralph

AU - Kathail, Vinod

AU - Gopalakrishnan, Padmini

AU - Pareek, Satyaprakash

AU - Jain, Rishabh

AU - Kandemir, Mahmut Taylan

AU - Lin, Jun Liang

AU - Akbulut, Gulsum Gudukbay

AU - Das, Chita R.

PY - 2024

Y1 - 2024

N2 - Deep learning recommendation models (DLRMs) are deployed extensively to support personalized recommendations and consume a large fraction of artificial intelligence (AI) cycles in modern datacenters with embedding stage being a critical component. Modern CPUs execute a lot of DLRM cycles because they are cost effective compared to GPUs and other accelerators. Our paper addresses key bottlenecks in accelerating the embedding stage on CPUs. Specifically, this work 1) explores novel threading schemes that parallelize embedding bag, 2) pushes the envelope on realized bandwidth by improving data reuse in caches, and 3) studies the impact of parallelization on load imbalance. The new embedding bag kernels have been prototyped in the ZenDNN software stack. When put together, our work on fourth generation EPYC processors achieve up to 9.9x improvement in embedding bag performance over state-of-the-art implementations, and improve realized bandwidth of up to 5.7x over DDR bandwidth.

AB - Deep learning recommendation models (DLRMs) are deployed extensively to support personalized recommendations and consume a large fraction of artificial intelligence (AI) cycles in modern datacenters with embedding stage being a critical component. Modern CPUs execute a lot of DLRM cycles because they are cost effective compared to GPUs and other accelerators. Our paper addresses key bottlenecks in accelerating the embedding stage on CPUs. Specifically, this work 1) explores novel threading schemes that parallelize embedding bag, 2) pushes the envelope on realized bandwidth by improving data reuse in caches, and 3) studies the impact of parallelization on load imbalance. The new embedding bag kernels have been prototyped in the ZenDNN software stack. When put together, our work on fourth generation EPYC processors achieve up to 9.9x improvement in embedding bag performance over state-of-the-art implementations, and improve realized bandwidth of up to 5.7x over DDR bandwidth.

UR - http://www.scopus.com/inward/record.url?scp=85200810872&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85200810872&partnerID=8YFLogxK

U2 - 10.1109/MM.2024.3423785

DO - 10.1109/MM.2024.3423785

M3 - Article

AN - SCOPUS:85200810872

SN - 0272-1732

VL - 44

SP - 44

EP - 51

JO - IEEE Micro

JF - IEEE Micro

IS - 6

ER -

Parallelization Strategies for DLRM Embedding Bag Operator on AMD CPUs

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this