Heterogeneous integration for artificial intelligence: Challenges and opportunities

The recent progress in artificial intelligence (AI) and machine learning (ML) has enabled computing platforms to solve highly complex difficult problems in computer vision, robotics, finance, security, and science. The algorithmic progress in AI/ML have motivated new research in hardware accelerators. The dedicated accelerators promise high energy efficiency compared to software solutions using CPU. However, as AI/ML models become complex, the increasing memory demands and, hence, high energy/time cost of communication between logic and memory possess a major challenge to energy efficiency. We review the potential of heterogeneous integration in addressing the preceding challenge and present different approaches to leverage heterogeneous integration for energy-efficient AI platforms. First, we discuss packaging technologies for efficient chip-to-chip communication. Second, we present near-memory-processing architecture for AI accelerations that leverages 3D die-stacking. Third, processing-in-memory architectures using heterogeneous integration of CMOS and embedded non-volatile memory are presented. Finally, the article presents case studies that integrate preceding concepts to advance AI/ML hardware platform for different application domains.