... Google is stupid for doing 8 generations of TPUs for their internal AI training & inference.
Analysis: AI will require $2T in annual revenue to support $500B in planned CapEx
- Thread starter Xebec
- Start date
... Google is stupid for doing 8 generations of TPUs for their internal AI training & inference.
I disagree. What is Google's fully-loaded cost per TPU chip, including Broadcom's cost? Even if it's $3000, and I doubt it's that high, how much would Nvidia GPUs cost? Nvidia's gross margin is over 70%. And if Google uses Nvidia GPUs they're locked into CUDA, so they can't innovate in the interface layers, and they can't innovate in the hardware architecture either. I think doing TPUs is a very smart move on Google's part, and Google has more than sufficient design expertise.
And, IMO, Amazon is very smart too designing Trainium and Inferentia.
Last edited:
FYI -- Broadcom's gross margin is higher than Nvidia's.
Not in the last quarter, not that it matters for this question, 67% versus 72.4%. But the real issue is Google's cost per TPU (which they don't publish) versus Nvidia GPUs. I can't believe Nvidia's price per GPU is lower than Google's overall cost per TPU. Not even close.
siliconbruh999
Well-known member
Google Uses TPU for their models training or inference.
According to co-pilot:
Broadcom reported a Q3 2025 gross margin of 78.4%, marking a significant year-over-year increase. This strong margin was driven by ...
Do the math yourself:
Anyway, it's irrelevant to my reasoning. The key point is that Google's TPU production costs are lower than Nvidia's prices, and Google gets various architectural and other cost benefits (like their integration of their own proprietary networks for scale-up and scale-out.)
Broadcom has a big software component now; corp. level financial metrics may not be comparable.
As to why Google is not commercializing their TPU, maybe they like their secret sauce? Maybe they consider it a moat?
I see the TPU as the only serious data center hardware from a hyperscaler. Google TPUs are only one component of entire large scale systems designed solely for AI. Most other folks, like Amazon, are dropping AI chips/trays into their more general purpose racks and networking. I'm not sure that can work for the long haul. The issue for Google TPU is mostly how can they monetize ?
Who would buy it ? And would they buy the chips, the racks, or what ? TPUs aren't useful without the whole superPod environment. Ultimately, it seems like the only way they can sell is via cloud AI, but they are at pretty small scale for that compared to the other hyperscalers and frontier AI model guys.
If Google wanted to sell the TPU as a chip they would have to also commercialize their proprietary networks. IMO, that would be a non-starter, for two reasons. Their networks are proprietary advantages to their cloud businesses, and it is doubtful anyone would agree to the proprietary lock-in. Also, Google does not have any business infrastructure to sell chips.
The other alternative would be to integrate Ethernet into their chips, and Ethernet is an inferior solution.
I'm skeptical of this. AWS is ruthlessly cost-sensitive. If they weren't getting substantial financial benefits from Trainium and Inferentia they'd kill the programs. I'm pretty confident of that.
This is silly. The numbers were reported in the 2025 3Q Earnings Report. It is typically frowned upon for CEOs & companies to lie about results like gross margin.
We'll have to see how Amazon does with Rainier and Anthropic.
The tiny chips behind Amazon’s big AI investment
Anthropic prepares to put the company’s Project Rainier, potentially the world’s most powerful computer, to the test.
www.semafor.com
Amazon has to do far better than early 2024, when they weren't even doing well with their own chips inside of they own customer base.
Business Insider, citing internal documents, said Amazon has struggled to find customers for its chips.
“Last year, the adoption rate of Trainium chips among AWS’s largest customers was just 0.5% of Nvidia’s GPUs, according to one of the internal documents. This assessment, which measures usage levels of different AI chips through AWS’s cloud service, was prepared in April 2024. Inferentia, another AWS chip designed for a type of AI task known as inference, was only slightly better, at 2.7% of the Nvidia usage rate.”
The Arm CPUs and AI processors from all three US cloud vendors are aimed largely at internal applications. These numbers are no surprise.We'll have to see how Amazon does with Rainier and Anthropic.
The tiny chips behind Amazon’s big AI investment
Anthropic prepares to put the company’s Project Rainier, potentially the world’s most powerful computer, to the test.
Amazon has to do far better than early 2024, when they weren't even doing well with their own chips inside of they own customer base.
Business Insider, citing internal documents, said Amazon has struggled to find customers for its chips.
“Last year, the adoption rate of Trainium chips among AWS’s largest customers was just 0.5% of Nvidia’s GPUs, according to one of the internal documents. This assessment, which measures usage levels of different AI chips through AWS’s cloud service, was prepared in April 2024. Inferentia, another AWS chip designed for a type of AI task known as inference, was only slightly better, at 2.7% of the Nvidia usage rate.”
Ole Hoejlund
New member
I agree that the demand will soften at some time. But I do not think you can analyze the AI market in isolation.
Of course, Google, Apple, Meta etc. invest because of the additional revenue streams that may open.
But in my opinion the primary reason is to protect and develop existing business models. Fx Google cannot afford to loose their ad business and they need to be on the forefront of search, programmatic advertising etc. to hold on to that.
Of course, Google, Apple, Meta etc. invest because of the additional revenue streams that may open.
But in my opinion the primary reason is to protect and develop existing business models. Fx Google cannot afford to loose their ad business and they need to be on the forefront of search, programmatic advertising etc. to hold on to that.