A Post-AI-ROI-Panic Overview of the Data Center Processing...

With all the Q2-24 results delivered, it is time to remove the clouds of euphoria and panic, ignore the performance claims and the bugs, and analyse the Data Center business, including examining the supply chain up and downstream. It is time to find out if the AI boom in semiconductors is still alive.

We begin the analysis with the two main categories, processing and network, and the top 5 semiconductor companies supplying the data center.

The top 5 Semiconductor companies that supply the data center account for nearly 100% of networking and processing. Once again, the overall growth for Q2-24 was a healthy 15.3%, all coming from processing. Networking contracted slightly by -2.5%, while processing grew by 20.3%. As Nvidia stated that the company’s decline in networking business was due to shipment adjustments, the growth numbers likely do not represent a major shift in the underlying business.

From a Year-over-year perspective, the overall growth was massive, 167%, with processing growing by 211% and networking by 66%.

As can be seen from the Operating Profit graph, the operating profit growth was much more aggressive, highlighting the massive demand from Data centers for Nvidia in particular.

The combined annual Operating profit growth was 522%, with processing accounting for a whopping 859% and networking growing 211%.

The quarterly operating profit growth rates aligned with the revenue growth rates, indicating that operating profits have stabilised and favour Processing slightly, as seen below.

Companies and Market shares

Even though Nvidia is so far ahead that market share is irrelevant for the GPU giant, it is vital for the other suitors. Every % is important.

The combined Datacenter processing revenue and market shares can be seen below:

While Nvidia has a strong revenue share of the total market, it has a complete stranglehold on the profits. The ability to get a higher premium is an important indicator of the width of the Nvidia moat. Nvidia’s competitors are trying to push performance/price metrics to convince customers to switch, but at the same time, they are commending Nvidia AI GPUs as Nvidia is running with a higher margin.

The shift in Market share can be seen below:

While this is “Suddenly, Nothing Happened,” the key takeaway is that despite the huff and puff from the other AI suitors, Nvidia stands firm and has slightly tightened its grip on profits.

The noise around Blackwell’s delay has not yet impacted the numbers, and it is doubtful that it will hurt Nvidia’s numbers, as the H100 is still the de facto choice in the data center.

The Datacenter Supply Chain

The shift in the Semiconductor market towards AI GPUs has significantly changed the Semiconductor supply chain. AI companies are now transforming into systems companies that control other parts of the supply chain, such as memory supply.

The supply situation is mostly unchanged from last quarter, with high demand from cloud companies and supply limited by CoWoS packaging and HBM memory. The memory situation is improving, although not all suppliers are approved by Nvidia.

As can be seen, the memory companies have been the clear winners in revenue growth since the last low point in the cycle.

Undoubtedly, SK Hynix has been the prime supplier to Nvidia, as Samsung has had approval problems. The latest operating profit data for Samsung suggest that the company is now delivering HBM to Nvidia or other companies, and the HMB supply situation is likely more relaxed.

GPU/CPU Supply

TSMC manufactures almost all of the processing and networking dies. The company recently reported record revenue for Q2-24 but is not yet at maximum capacity. CoWoS is the only area that is still limited, but TSMC is adding significant capacity every quarter, and it will not impact the key players in the Data Center supply chain.

Also, the monthly revenue for July was a new record.

While nothing has been revealed about the July revenue, it is likely still driven by TSMC’s High-Performance Computing business, which supplies mainly to the data center.

The HPC business added $3B without revealing the customer or the background of the company. As Apple used to be the only 3nm customer and normally buys less in Q2, it looks like it is a new 3nm customer and that is most likely a Datacenter supplier.

It could be one of the Cloud Companies that all are trying to leverage their own architectures. Amazon is very active with Trainium, Inferential and Graviton while Google has the TPU.

Also, Lunar Lake from Intel and the MI series from AMD could be candidates. With Nvidia’s Blackwell issues, the company stays on 4nm (5nm) until Rubin is ready to launch.

The possibility of Apple starting using M-series processors in their data centers is also possible.

The TSMC revenue increase is undoubtedly good news for the Data center market, which will continue growing in Q3, no matter what opinions investment banks have on the ROI of AI.

The Demand Side of the Equation

The AI revolution has caused the explosive growth in Data center computing. Analysing Nvidia’s current customer base, gives an idea of the different demand channels driving the growth.

2/3 of the demand is driven by the large Tech companies in Cloud and consumer, where the last third is more fragmented around Enterprise, Sovereign and Supercomputing. The last two are not really driven from a short term ROI perspective and will not suddenly disappear.

A number of Banks and financial institutions have recently questioned the investment of the large tech companies into AI which have cause the recent bear run in the stock market.

I am not one to run with conspiracy theories but it is well known that volatility is good for banking business. I also know that the banks have no clue about the long term return in AI, just like me, so I will continue to follow the facts, while the markets go up and down.

The primary source of funding for the AI boom will continue to be the large tech companies

Tech CapEx

5 companies represent the bulk of the CapEx that flows to the Data Center Processing market.

It is almost like the financial community treats the entire CapEx for the large Cloud customers as a brand new investment into a doubtfull AI business model. The reality is that the Datacenter investment is not new and it is creating tangible revenue streams at the same time as doubling as an AI investment.

From a growth perspective and using a startpoint from before the AI boom, it becomes clear that the datacenter investment growth actually follows the growth of the cloud revenue growth.

While I will let other people decide if that is a good return on investment, the CapEx growth compared to Cloud revenue growth does not look insane. That might happen later but right now it can certainly be defended.

The next question is, how much processing candy the large cloud companies can get for their CapEx?

The processing share of total capex is certainly increasing although the capex also have increased significantly since the AI boom started. It is worth noting that the new AI servers delivers significantly more performance that the earlier CPU only servers that traditionally has been used in the data centers.

The Q2 increase in CapEx is a good sign for the Datacenter Processing companies. It represents a 8.3B$ increase in CapEx for top 5. This can be compared with a 4.3B$ increase in Processing and Networking revenue for the Semiconductor companies.

What is even better is that the CapEx commitment from the large cloud companies will continue for the foreseeable future. Alphabet, Meta and Amazon will have higher CapEx budgets in 2nd half and Meta will have significantly higher CapEx in 2025.

Microsoft revealed that even though almost all the CapEx is AI and data center related, around half of the current CapEx is used for land and buildings. These are boxes that needs to be filled with loads of expensive AI GPU servers later and a strong commitment to long term CapEx.

Conclusion

While the current valuations and share price fluctuations might be insane, the Semiconductor side of the equation is high growth but not crazy. It is alive and vibrant.

Nvidia might have issues with Blackwell but can keep selling H100 instead. AMD and Intel will start to chip away at Nvidia but it has not happened yet. Cloud companies will also start to sneak in their architectures.

The supply chain looks better aligned to serve the new AI driven business with improved supply of memory although advanced packaging might be tight still.

TSMC has rapidly increasing HPC revenue that is a good sign for the next revenue season.

The CapEx from the large cloud companies is growing in line with their cloud revenue and all have committed to strong CapEx budgets for the next 2 to 6 quarters.

In a few weeks, Nvidia will start the Data Center Processing earnings circus once again. I will have my popcorn ready.

In the Meta call, the ROI on AI was addressed with two buckets: Core AI where a ROI view is relevant and a Gen AI that is a long term bet where ROI does not make sense to talk about yet.