Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization

N. Gong; M. J. Rasch; S. C. Seo; A. Gasasira; P. Solomon; V. Bragaglia; S. Consiglio; H. Higuchi; C. Park; K. Brew; P. Jamison; C. Catano; I. Saraf; F. F. Athena; C. Silvestre; X. Liu; B. Khan; N. Jain; S. McDermott; R. Johnson; I. Estrada-Raygoza; J. Li; T. Gokmen; N. Li; R. Pujari; F. Carta; H. Miyazoe; M. M. Frank; D. Koty; Q. Yang; R. Clark; K. Tapily; C. Wajda; A. Mosden; J. Shearer; A. Metz; S. Teehan; N. Saulnier; B. J. Offrein; T. Tsunomura; G. Leusink; V. Narayanan; T. Ando

doi:10.1109/IEDM45625.2022.10019569

Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization

N. Gong, M. J. Rasch, S. C. Seo, A. Gasasira, P. Solomon, V. Bragaglia, S. Consiglio, H. Higuchi, C. Park, K. Brew, P. Jamison, C. Catano, I. Saraf, F. F. Athena, C. Silvestre, X. Liu, B. Khan, N. Jain, S. McDermott, R. JohnsonI. Estrada-Raygoza, J. Li, T. Gokmen, N. Li, R. Pujari, F. Carta, H. Miyazoe, M. M. Frank, D. Koty, Q. Yang, R. Clark, K. Tapily, C. Wajda, A. Mosden, J. Shearer, A. Metz, S. Teehan, N. Saulnier, B. J. Offrein, T. Tsunomura, G. Leusink, V. Narayanan, T. Ando

Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

12 Scopus citations

Abstract

We show for the first time in hardware that in contrast to conventional stochastic gradient descent (SGD), our modified SGD algorithm (TTv2) together with a co-optimized ReRAM material achieves respectable accuracy (98%) on reduced MNIST classification (0 & 1), approaching a floating point (FP) baseline. To extrapolate these insights towards larger DNN training workloads in simulations, we establish an analog switching test sequence and extract key device statistics from 6T1R ReRAM arrays (up to 2k devices) built on a 14nm CMOS baseline. With this, we find that for larger DNN workloads, device and algorithm co-optimization shows dramatic improvements in comparison to standard SGD and baseline ReRAM. The gap to the reference floating-point accuracy across various tested DNNs indicates that further material and algorithmic optimizations are still needed. This work shows a pathway for scalable in-memory deep learning training using ReRAM crossbar arrays.

Original language	English (US)
Title of host publication	2022 International Electron Devices Meeting, IEDM 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3371-3374
Number of pages	4
ISBN (Electronic)	9781665489591
DOIs	https://doi.org/10.1109/IEDM45625.2022.10019569
State	Published - 2022
Event	2022 International Electron Devices Meeting, IEDM 2022 - San Francisco, United States Duration: Dec 3 2022 → Dec 7 2022

Publication series

Name	Technical Digest - International Electron Devices Meeting, IEDM
Volume	2022-December
ISSN (Print)	0163-1918

Conference

Conference	2022 International Electron Devices Meeting, IEDM 2022
Country/Territory	United States
City	San Francisco
Period	12/3/22 → 12/7/22

All Science Journal Classification (ASJC) codes

Electronic, Optical and Magnetic Materials
Condensed Matter Physics
Electrical and Electronic Engineering
Materials Chemistry

Access to Document

10.1109/IEDM45625.2022.10019569

Cite this

Gong, N., Rasch, M. J., Seo, S. C., Gasasira, A., Solomon, P., Bragaglia, V., Consiglio, S., Higuchi, H., Park, C., Brew, K., Jamison, P., Catano, C., Saraf, I., Athena, F. F., Silvestre, C., Liu, X., Khan, B., Jain, N., McDermott, S., ... Ando, T. (2022). Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. In 2022 International Electron Devices Meeting, IEDM 2022 (pp. 3371-3374). (Technical Digest - International Electron Devices Meeting, IEDM; Vol. 2022-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IEDM45625.2022.10019569

@inproceedings{4d4a26fea3404a61a40d036127e043ed,

title = "Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization",

abstract = "We show for the first time in hardware that in contrast to conventional stochastic gradient descent (SGD), our modified SGD algorithm (TTv2) together with a co-optimized ReRAM material achieves respectable accuracy (98%) on reduced MNIST classification (0 & 1), approaching a floating point (FP) baseline. To extrapolate these insights towards larger DNN training workloads in simulations, we establish an analog switching test sequence and extract key device statistics from 6T1R ReRAM arrays (up to 2k devices) built on a 14nm CMOS baseline. With this, we find that for larger DNN workloads, device and algorithm co-optimization shows dramatic improvements in comparison to standard SGD and baseline ReRAM. The gap to the reference floating-point accuracy across various tested DNNs indicates that further material and algorithmic optimizations are still needed. This work shows a pathway for scalable in-memory deep learning training using ReRAM crossbar arrays.",

author = "N. Gong and Rasch, {M. J.} and Seo, {S. C.} and A. Gasasira and P. Solomon and V. Bragaglia and S. Consiglio and H. Higuchi and C. Park and K. Brew and P. Jamison and C. Catano and I. Saraf and Athena, {F. F.} and C. Silvestre and X. Liu and B. Khan and N. Jain and S. McDermott and R. Johnson and I. Estrada-Raygoza and J. Li and T. Gokmen and N. Li and R. Pujari and F. Carta and H. Miyazoe and Frank, {M. M.} and D. Koty and Q. Yang and R. Clark and K. Tapily and C. Wajda and A. Mosden and J. Shearer and A. Metz and S. Teehan and N. Saulnier and Offrein, {B. J.} and T. Tsunomura and G. Leusink and V. Narayanan and T. Ando",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 International Electron Devices Meeting, IEDM 2022 ; Conference date: 03-12-2022 Through 07-12-2022",

year = "2022",

doi = "10.1109/IEDM45625.2022.10019569",

language = "English (US)",

series = "Technical Digest - International Electron Devices Meeting, IEDM",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3371--3374",

booktitle = "2022 International Electron Devices Meeting, IEDM 2022",

address = "United States",

}

Gong, N, Rasch, MJ, Seo, SC, Gasasira, A, Solomon, P, Bragaglia, V, Consiglio, S, Higuchi, H, Park, C, Brew, K, Jamison, P, Catano, C, Saraf, I, Athena, FF, Silvestre, C, Liu, X, Khan, B, Jain, N, McDermott, S, Johnson, R, Estrada-Raygoza, I, Li, J, Gokmen, T, Li, N, Pujari, R, Carta, F, Miyazoe, H, Frank, MM, Koty, D, Yang, Q, Clark, R, Tapily, K, Wajda, C, Mosden, A, Shearer, J, Metz, A, Teehan, S, Saulnier, N, Offrein, BJ, Tsunomura, T, Leusink, G, Narayanan, V & Ando, T 2022, Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. in 2022 International Electron Devices Meeting, IEDM 2022. Technical Digest - International Electron Devices Meeting, IEDM, vol. 2022-December, Institute of Electrical and Electronics Engineers Inc., pp. 3371-3374, 2022 International Electron Devices Meeting, IEDM 2022, San Francisco, United States, 12/3/22. https://doi.org/10.1109/IEDM45625.2022.10019569

Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. / Gong, N.; Rasch, M. J.; Seo, S. C. et al.
2022 International Electron Devices Meeting, IEDM 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 3371-3374 (Technical Digest - International Electron Devices Meeting, IEDM; Vol. 2022-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Deep learning acceleration in 14nm CMOS compatible ReRAM array

T2 - 2022 International Electron Devices Meeting, IEDM 2022

AU - Gong, N.

AU - Rasch, M. J.

AU - Seo, S. C.

AU - Gasasira, A.

AU - Solomon, P.

AU - Bragaglia, V.

AU - Consiglio, S.

AU - Higuchi, H.

AU - Park, C.

AU - Brew, K.

AU - Jamison, P.

AU - Catano, C.

AU - Saraf, I.

AU - Athena, F. F.

AU - Silvestre, C.

AU - Liu, X.

AU - Khan, B.

AU - Jain, N.

AU - McDermott, S.

AU - Johnson, R.

AU - Estrada-Raygoza, I.

AU - Li, J.

AU - Gokmen, T.

AU - Li, N.

AU - Pujari, R.

AU - Carta, F.

AU - Miyazoe, H.

AU - Frank, M. M.

AU - Koty, D.

AU - Yang, Q.

AU - Clark, R.

AU - Tapily, K.

AU - Wajda, C.

AU - Mosden, A.

AU - Shearer, J.

AU - Metz, A.

AU - Teehan, S.

AU - Saulnier, N.

AU - Offrein, B. J.

AU - Tsunomura, T.

AU - Leusink, G.

AU - Narayanan, V.

AU - Ando, T.

PY - 2022

Y1 - 2022

N2 - We show for the first time in hardware that in contrast to conventional stochastic gradient descent (SGD), our modified SGD algorithm (TTv2) together with a co-optimized ReRAM material achieves respectable accuracy (98%) on reduced MNIST classification (0 & 1), approaching a floating point (FP) baseline. To extrapolate these insights towards larger DNN training workloads in simulations, we establish an analog switching test sequence and extract key device statistics from 6T1R ReRAM arrays (up to 2k devices) built on a 14nm CMOS baseline. With this, we find that for larger DNN workloads, device and algorithm co-optimization shows dramatic improvements in comparison to standard SGD and baseline ReRAM. The gap to the reference floating-point accuracy across various tested DNNs indicates that further material and algorithmic optimizations are still needed. This work shows a pathway for scalable in-memory deep learning training using ReRAM crossbar arrays.

AB - We show for the first time in hardware that in contrast to conventional stochastic gradient descent (SGD), our modified SGD algorithm (TTv2) together with a co-optimized ReRAM material achieves respectable accuracy (98%) on reduced MNIST classification (0 & 1), approaching a floating point (FP) baseline. To extrapolate these insights towards larger DNN training workloads in simulations, we establish an analog switching test sequence and extract key device statistics from 6T1R ReRAM arrays (up to 2k devices) built on a 14nm CMOS baseline. With this, we find that for larger DNN workloads, device and algorithm co-optimization shows dramatic improvements in comparison to standard SGD and baseline ReRAM. The gap to the reference floating-point accuracy across various tested DNNs indicates that further material and algorithmic optimizations are still needed. This work shows a pathway for scalable in-memory deep learning training using ReRAM crossbar arrays.

UR - http://www.scopus.com/inward/record.url?scp=85147533406&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85147533406&partnerID=8YFLogxK

U2 - 10.1109/IEDM45625.2022.10019569

DO - 10.1109/IEDM45625.2022.10019569

M3 - Conference contribution

AN - SCOPUS:85147533406

T3 - Technical Digest - International Electron Devices Meeting, IEDM

SP - 3371

EP - 3374

BT - 2022 International Electron Devices Meeting, IEDM 2022

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 3 December 2022 through 7 December 2022

ER -

Gong N, Rasch MJ, Seo SC, Gasasira A, Solomon P, Bragaglia V et al. Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. In 2022 International Electron Devices Meeting, IEDM 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 3371-3374. (Technical Digest - International Electron Devices Meeting, IEDM). doi: 10.1109/IEDM45625.2022.10019569

Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this