Synopsys Panel Updates on the State of Multi-Die Systems

Synopsys recently hosted a cross-industry panel on the state of multi-die systems which I found interesting not least for its relevance to the rapid acceleration in AI-centric hardware. More on that below. Panelists, all with significant roles in multi-die systems, were Shekhar Kapoor (Senior Director of Product Management, Synopsys), Cheolmin Park (Corporate VP, Samsung), Lalitha Immaneni (VP Architecture, Design and Technology Solutions, Intel), Michael Schaffert (Senior VP, Bosch), and Murat Becer (VP R&D, Ansys). The panel was moderated by Marco Chiappetta (Co-Founder and Principal Analyst, HotTech Vision and Analysis).

A Big Demand Driver

It is common under this heading to roll out all the usual suspects (HPC, Automotive, etc) but that list sells short maybe the biggest underlying factor – the current scramble for dominance in everything LLM and generative AI. Large language models offer new levels of SaaS services in search, document creation and other capabilities, with major competitive advantages to whoever gets this right first. On mobile devices and in the car, superior natural language-based control and feedback will make existing voice-based options look primitive by comparison. Meanwhile generative methods for creating new images using Diffusion and Poisson flow models can pump out spectacular graphics drawing on text or a photograph complemented by image libraries. As a consumer draw this could prove to be the next big thing for future phone releases.

While transformer-based AI presents a huge $$$ opportunity it comes with challenges. The technologies that make such methods possible are already proven in the cloud and emerging at the edge, yet they are famously memory hungry. Production LLMs run anywhere from billions to trillions of parameters which must be loaded to the transformer. Demand for in-process workspace is equally high; diffusion-based imaging progressively adds noise to a full image then works its way back to a modified image, again through transformer-based platforms.

Apart from an initial load, none of these processes can afford the overhead of interacting with external DRAM. Latencies would be unacceptable and power demand would drain a phone battery or would blow the power budget for a datacenter. All the memory needs to be near – very near – the compute. One solution is to stack SRAM on top of the accelerator (as AMD and now Intel have demonstrated for their server chips). High bandwidth memory in-package adds another somewhat slower option but still not as slow as off-chip DRAM.

All of which requires multi-die systems. So where are we at in making that option production-ready?

Views on where we are at

I heard a lot of enthusiasm for growth in this domain, in adoption, applications and tooling. Intel, AMD, Qualcomm, Samsung are all clearly very active in this space. Apple M2 Ultra is known to be a dual die design, and AWS Graviton 3 a multi-die system. I am sure there are plenty of other examples among the big systems and semiconductor houses. I get the impression that die are still sourced predominantly internally (except perhaps for HBM stacks), and assembled in foundry packaging technologies from TSMC, Samsung or Intel. However, Tenstorrent just announced that they have chosen Samsung to manufacture their next generation AI design as a chiplet (a die suitable to be used in a multi-die system), so this space is already inching to towards broader die sourcing.

All panelists were naturally enthusiastic about the general direction, and clearly technologies and tools are evolving fast which accounts for the buzz. Lalitha grounded that enthusiasm by noting that the way that multi-die systems currently being architected and designed is still in its infancy, not yet ready to launch an extensive reusable market for die. That doesn’t surprise me. Technology of this complexity seems like it should mature first in tight partnerships between system designers, foundries and EDA companies, maybe over several years before it can extend to a larger audience.

I’m sure that foundries, system builders and EDA companies aren’t showing all their cards and may be further along than they choose to advertise. I look forward to hearing more. You can watch the panel discussion HERE.