You are currently viewing SemiWiki as a guest which gives you limited access to the site. To view blog comments and experience other SemiWiki features you must be a registered member. Registration is fast, simple, and absolutely free so please, join our community today!
From data centers to autonomous vehicles, discover how Micron's broad portfolio of #ai solutions collectively contributes to shaping the future of AI, powering innovation across all sectors. Today’s generative AI models require an ever-growing amount of data as they scale to deliver better results and address new opportunities.
Micron’s 1-beta memory technology leadership and packaging advancements ensure the most efficient data flow in and out of the GPU. Micron’s 8-high and 12-high #HBM3E memory further fuel AI innovation at 30% lower power consumption than the competition. The 8-high 24GB solution will be part of #nvidia H200 Tensor Core GPUs, which will begin shipping in the second calendar quarter of 2024.
If we see leading edge AI models continue to progress the way they are going, with exponential growth in model parameter count, with reduced compute for a large percentage of the parameters (FP4, sparsity harvesting), we’re likely to see logic-in-memory approaches that offer lower power and less costly hardware.
If we see leading edge AI models continue to progress the way they are going, with exponential growth in model parameter count, with reduced compute for a large percentage of the parameters (FP4, sparsity harvesting), we’re likely to see logic-in-memory approaches that offer lower power and less costly hardware.
The problem with logic-in-memory, it was called computational memory for a long time, is more software than hardware. There are no standards for what the commands look like, how they're communicated on to the memory processing elements (e.g. write a command to a certain location, along with operands) and how the computational memory processing is extracted from popular programming languages. If you need specialized programming at the application level you're probably going to have a very limited market. These are messy problems. Even computational storage, which is a much easier problem, hasn't taken off after decades of talk and announcements, and even products. It is difficult to be hopeful.
