Lowering Latency of Embedded Memory by Exploiting In-Cell Victim Cache Hierarchy Based on Emerging Multi-Level Memory Devices

Juejian Wu, Tianyu Liao, Taixin Li, Yixin Xu, Vijaykrishnan Narayanan, Yongpan Liu, Huazhong Yang, Xueqing Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The concept of multi-level cell (MLC) enabled by emerging memory device technologies has introduced new opportunities for memory density improvement, including in the cache scenarios with some high-endurance technologies. However, the access latency of different bits within an MLC memory cell is inherently nonuniform, which raises challenges in utilizing the MLC technology for low-latency cache. To exploit the access performance of the MLC cache, the key is identifying the hot data blocks and mapping them to fast MLC bits. Prior works perform the hot/cold data management based on block-wise access patterns with considerable hardware overheads. Inspired by the memory hierarchy, this work proposes a new concept of in-cell hierarchical victim cache as embedded memory and systematically presents the cache architecture, operating mechanism, design space exploration, optimizations, and evaluations. By utilizing the slow bits as the victim buffer, lower hit latency with low implementation overheads is achieved. Based on the in-cell victim cache, two optimization techniques, namely selective victim retrieval, and victim-bypassing write (VBW) are proposed, to further improve performance and prolong cache endurance, respectively. Evaluation results show that the MLC victim cache significantly improves the average system performance by 20.2% over conventional MLC cache and achieves 98% performance of the upper bound implemented with 2x memory cells SLC. The proposed VBW also reduces energy consumption by 21% and improves lifetime by over 80%, showing a new promising dimension for future MLC cache design.

Original languageEnglish (US)
Title of host publication2023 42nd IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2023 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350315592
StatePublished - 2023
Event42nd IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2023 - San Francisco, United States
Duration: Oct 28 2023Nov 2 2023

Publication series

NameIEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD
ISSN (Print)1092-3152


Conference42nd IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2023
Country/TerritoryUnited States
CitySan Francisco

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Science Applications
  • Computer Graphics and Computer-Aided Design

Cite this