3D chip design targets AI memory bottleneck

Engineers from Stanford University, Carnegie Mellon University, the University of Pennsylvania, and the Massachusetts Institute of Technology worked with SkyWater Technology to develop a multilayer 3D chip aimed at easing a core constraint in AI hardware: the slow movement of data between memory and compute units.

The prototype uses monolithic 3D integration, building each layer directly on top of the previous one rather than bonding separate chips together. By stacking ultra thin components vertically and adding dense vertical wiring, the design places memory and logic closer together and creates more routes for data to move, addressing both the “memory wall” and the limits of continued transistor miniaturization.

Early hardware tests showed the chip outperforming comparable 2D chips by about four times, while simulations suggest up to a twelve fold improvement on real AI workloads, including work derived from Meta’s open source LLaMA model. The researchers also point to a potential path toward 100 to 1,000 fold improvements in energy delay product.

All fabrication was completed at SkyWater Technology’s Bloomington, Minnesota, Foundry, which the team presented as evidence that advanced monolithic 3D chips can be produced domestically and at commercial scale.