Nvidia's Jensen Huang states we are going through two fundamental changes in Compute - Do you agree?

[Brassmonke]

During last nights conference call for Q4 earnings Jensen stated that there were two fundamental changes in compute. Obviously one of those changes was Generative AI but his second pivotal change, according to Jensen, was that compute has now moved from General to Accelerated computing. Throughout the call he continued to indicate that general compute was dead and that accelerated compute was the only way forwards and it felt as if he was saying that Moore's Law applied to General Compute (CPU driven compute) but not Accelerated or GPU compute. What is the communities read through on this? Part of the reason I ask is that the statement is self serving so I can't take Jensen's comment without a degree of skepticism. Is this a roundabout way of excusing Grace CPU performance?

The second and somewhat unrelated question is in regards to his comments around AI Enterprise, already at $1 billion run rate. We know that MSFT is now delivering oracle database services on oracle cloud hosted in MSFT Azure, the only non ORCL infrastructure to do so. Jensen said, in his commentary that there are a multitude of cloud service providers that are not the megacap names and, therefore, do not have their own engineering teams that would be able to enhance, maintain, patch their software for compatibility with the broad NVDA AI ecosystem. How accurate is this claim? Do you believe that may lead some cloud service providers to look towards alternatives to NVDA hosted enterprise in a similar fashion as ORCL has or am I way off base here? Would they even want to?

 

[blueone]

During last nights conference call for Q4 earnings Jensen stated that there were two fundamental changes in compute. Obviously one of those changes was Generative AI but his second pivotal change, according to Jensen, was that compute has now moved from General to Accelerated computing. Throughout the call he continued to indicate that general compute was dead and that accelerated compute was the only way forwards and it felt as if he was saying that Moore's Law applied to General Compute (CPU driven compute) but not Accelerated or GPU compute. What is the communities read through on this? Part of the reason I ask is that the statement is self serving so I can't take Jensen's comment without a degree of skepticism. Is this a roundabout way of excusing Grace CPU performance?

This has been the case for years. General-purpose CPUs are by no means dead, that's silly, but accelerators have been beating CPUs for their intended workloads for decades. Proof that this is the case is in Intel's own Xeon CPUs.

Intel® Accelerator Engines

Improve performance with cost and energy efficiency using Intel® Xeon® Scalable processor with integrated Intel® Accelerator Engines.

www.intel.com

With regards to Moore's Law not applying to accelerators, I haven't seen a quote from Jensen on the topic, but I don't believe that's what he said. The fundamental issue is that architecture (high-level chip architecture down to microarchitecture), hardware implementations of specialized functions, and parallelism are capable of providing more benefits than those from greater transistor density applied to general purpose CPUs. If that's what Jensen was referring to, I couldn't agree more. In Jensen's view, IMO, he's saying general purpose CPUs are being relegated to the highest-level software flows, and the most important work is being migrated to accelerators, like GPUs.

The second and somewhat unrelated question is in regards to his comments around AI Enterprise, already at $1 billion run rate. We know that MSFT is now delivering oracle database services on oracle cloud hosted in MSFT Azure, the only non ORCL infrastructure to do so. Jensen said, in his commentary that there are a multitude of cloud service providers that are not the megacap names and, therefore, do not have their own engineering teams that would be able to enhance, maintain, patch their software for compatibility with the broad NVDA AI ecosystem. How accurate is this claim? Do you believe that may lead some cloud service providers to look towards alternatives to NVDA hosted enterprise in a similar fashion as ORCL has or am I way off base here? Would they even want to?

Nvidia is claiming that because many cloud service providers do not have huge software engineering teams who develop vertical software stacks for their clouds, they will need Nvidia's AI Enterprise emerging software initiative to be competitive with the mega-companies, and that is a huge revenue opportunity for Nvidia, and I agree.

 

[nghanayem]

During last nights conference call for Q4 earnings Jensen stated that there were two fundamental changes in compute. Obviously one of those changes was Generative AI but his second pivotal change, according to Jensen, was that compute has now moved from General to Accelerated computing.

His firm's primary product is general purpose compute (even if he doesn't want to admit it). But if we pretend that a GP-GPU is a domain specific chip, then I would still disagree. Phones and PCs will always be dominated by GP SOCs albeit with an increasing number of application specific IP thrown in (think display engines, NPUs, image processing/upscalling engines, etc). Programmers are also something that is in critically short supply (just look at the insane comp at FANG type companies). General purpose compute is easier to develop for and also will have a wider software base to work off of (windows x86 ecosystem, Linux x86/ARM ecosystems, CUDA/omniverse, etc).

Throughout the call he continued to indicate that general compute was dead and that accelerated compute was the only way forwards and it felt as if he was saying that Moore's Law applied to General Compute (CPU driven compute) but not Accelerated or GPU compute. What is the communities read through on this? Part of the reason I ask is that the statement is self serving so I can't take Jensen's comment without a degree of skepticism. Is this a roundabout way of excusing Grace CPU performance?

That would be a bad take. Scaling isn't dead, even if NVIDIA would like to tell folks that is why the price of their products goes up every gen (and not because they increased their corporate GM). If we go back to assuming that a GP-GPU isn't GP compute, even then scaling is critical for NVIDIA. Currently they do monolithic designs that are die size limited. More scaling means they can throw more cores onto their chips and or improve the microarch itself without paying the power/area penalty they would have to eat if it was on the same old node. Once they go disag, lower Xtor power and denser inter die connections will allow for even more compute to be crammed into a given power envelop. In other words Huang's law is powered by better transistors. Without it NVIDIA wouldn't be able to improve as rapidly.

 

[Brassmonke]

This has been the case for years. General-purpose CPUs are by no means dead, that's silly, but accelerators have been beating CPUs for their intended workloads for decades. Proof that this is the case is in Intel's own Xeon CPUs.

Intel® Accelerator Engines

Improve performance with cost and energy efficiency using Intel® Xeon® Scalable processor with integrated Intel® Accelerator Engines.

www.intel.com

With regards to Moore's Law not applying to accelerators, I haven't seen a quote from Jensen on the topic, but I don't believe that's what he said. The fundamental issue is that architecture (high-level chip architecture down to microarchitecture), hardware implementations of specialized functions, and parallelism are capable of providing more benefits than those from greater transistor density applied to general purpose CPUs. If that's what Jensen was referring to, I couldn't agree more. In Jensen's view, IMO, he's saying general purpose CPUs are being relegated to the highest-level software flows, and the most important work is being migrated to accelerators, like GPUs.

Nvidia is claiming that because many cloud service providers do not have huge software engineering teams who develop vertical software stacks for their clouds, they will need Nvidia's AI Enterprise emerging software initiative to be competitive with the mega-companies, and that is a huge revenue opportunity for Nvidia, and I agree.

I appreciate the response. Jensen did not refer to Moore's law in any way. I suppose it was more the notion that its the natural progression in expanding the capabilities of compute; as the physical limitations of increasing chip density raise costs the industry seems to have moved to, as you have indicated, towards innovation in architecture and parallelism to push compute power per $ spent. As such, it seems you would agree, then, that innovation in architecture or accelerated compute/parallelism are going to be the source of future hardware innovation. As you said, the most important work is migrated there.
Perhaps I was being overly dramatic in saying general compute is dead. However, what then is the future for serial compute?

Nothing to note on your second observation. I would agree as well. Its seems they are aggressively broadening their software ecosystem to boot. I love that excess capital also being send to incubate future software that will end up on their platform anyway.

 

[KevinK]

I can see two things happening: 1) We‘re seeing the creation of a new app category, generative AI, that greatly benefits from a fundamentally different hardware computing architecture, mix of resources and ecosystem vs. traditional servers and 2) enough money chasing that app category that companies are choosing a substantially different allocation of resources to pursue that app category. I’m thinking that we’re seeing a split in the server business along the lines of the split we saw in the client side business between PC and smartphone, except the new entrant is at the high end, as opposed to the low end.

I thought this line from Jensen was especially telling - that’s like a 50% cut in spending on general purpose servers and infrastructure, if true broadly for CSPs.

And you can tell by the CSPs extending and many data centers, including our own for general-purpose computing, extending the depreciation from four to six years.

Seems like there is a major long-term reallocation of existing capital, plus new money flowing into the AI app category.