Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

[soAsian]

Pivot will hinge on success of next-gen Maia accelerator[/HEADING]

Thu 2 Oct 2025 // 20:41 UTC

Microsoft buys a lot of GPUs from both Nvidia and AMD. But moving forward, Redmond's leaders want to shift the majority of its AI workloads from GPUs to its own homegrown accelerators.

The software titan is rather late to the custom silicon party. While Amazon and Google have been building custom CPUs and AI accelerators for years, Microsoft only revealed its Maia AI accelerators in late 2023.

Driving the transition is a focus on performance per dollar, which for a hyperscale cloud provider is arguably the only metric that really matters. Speaking during a fireside chat moderated by CNBC on Wednesday, Microsoft CTO Kevin Scott said that up to this point, Nvidia has offered the best price-performance, but he's willing to entertain anything in order to meet demand.

Going forward, Scott suggested Microsoft hopes to use its homegrown chips for the majority of its datacenter workloads.

When asked, "Is the longer term idea to have mainly Microsoft silicon in the data center?" Scott responded, "Yeah, absolutely."

Later, he told CNBC, "It's about the entire system design. It's the networks and cooling, and you want to be able to have the freedom to make decisions that you need to make in order to really optimize your compute for the workload."

With its first in-house AI accelerator, the Maia 100, Microsoft was able to free up GPU capacity by shifting OpenAI's GPT-3.5 to its own silicon back in 2023. However, with just 800 teraFLOPS of BF16 performance, 64GB of HBM2e, and 1.8TB/s of memory bandwidth, the chip fell well short of competing GPUs from Nvidia and AMD.

Microsoft is reportedly in the process of bringing a second-generation Maia accelerator to market next year that will no doubt offer more competitive compute, memory, and interconnect performance.

But while we may see a change in the mix of GPUs to AI ASICs in Microsoft data centers moving forward, they're unlikely to replace Nvidia and AMD's chips entirely.

Over the past few years, Google and Amazon have deployed tens of thousands of their TPUs and Trainium accelerators. While these chips have helped them secure some high-profile customer wins, Anthropic for example, these chips are more often used to accelerate the company's own in-house workloads.

As such, we continue to see large-scale Nvidia and AMD GPU deployments on these cloud platforms, in part because customers still want them.

It should be noted that AI accelerators aren't the only custom chips Microsoft has been working on. Redmond also has its own CPU called Cobalt and a whole host of platform security silicon designed to accelerate cryptography and safeguard key exchanges across its vast datacenter domains. ®

Microsoft aims to swap AMD, Nvidia GPUs for its own AI chips

: Pivot will hinge on success of next-gen Maia accelerator

www.theregister.com

 

[siliconbruh999]

He can dream

 

[Xebec]

He can dream

Microsoft, brought to you by Huawei...

 

[blueone]

I think it's a worthwhile strategy, and Microsoft is trailing Google and Amazon in implementing it. If your real objective is best price-performance, not necessarily best performance, and your volumes are high enough to offset the costs, the results can be very profitable. Especially for in-house applications. Meta just bought Rivos for just this reason.

 

[KevinK]

He can dream

I think there are two very fundamental problems with their thinking per his statement in the article:

Later, he told CNBC, "It's about the entire system design. It's the networks and cooling, and you want to be able to have the freedom to make decisions that you need to make in order to really optimize your compute for the workload."

* Microsoft isn't in a position where they can build customized chips for all the needed optimized element for the entire system - some parts yes, but nowhere near what would likely be required.
* How much of their data center pools of workloads is actually theirs, vs OpenAI's vs others.

We'll see how things play out, but I'm a skeptical about in-house chips for every hyperscaler / CSP, especially when they have a huge pools of "legacy workloads" and data centers.

 

[blueone]

I think there are two very fundamental problems with their thinking per his statement in the article:

Later, he told CNBC, "It's about the entire system design. It's the networks and cooling, and you want to be able to have the freedom to make decisions that you need to make in order to really optimize your compute for the workload."

* Microsoft isn't in a position where they can build customized chips for all the needed optimized element for the entire system - some parts yes, but nowhere near what would likely be required.
* How much of their data center pools of workloads is actually theirs, vs OpenAI's vs others.

We'll see how things play out, but I'm a skeptical about in-house chips for every hyperscaler / CSP, especially when they have a huge pools of "legacy workloads" and data centers.

Azure is the #2 cloud computing company in the world. Their proprietary cloud-native applications include six database management systems (and Oracle on Azure, which uses Azure storage), four distributed storage systems I'm aware of (including block, file, and multiple blob stores), and all of the Microsoft applications are available in cloud-native form, like the entire Outlook suite, Teams, Office... Microsoft's internal cloud-native application environment is huge. I would guess millions of server CPUs are consumed running all of these applications. And the CPUs and their entire software stacks are completely invisible to the application users. Just like AWS. Just like Google Cloud, but Azure is bigger, and Microsoft apps are considered more enterprise level than Google's.

Unlike Amazon and Google, Microsoft has resisted getting into the chip design business. When they wanted more control and offload of their network processing, they tried a custom superNIC based on FPGAs to avoid chip design:

Azure SmartNIC - Microsoft Research

Azure Accelerated Networking (AccelNet) is our solution for offloading host networking to hardware, using custom Azure SmartNICs based on FPGAs.

www.microsoft.com

Microsoft likes to brag about the FPGA NIC, but then they acquired Fungible in 2023 for $190M. That looks like a course correction to me. Fungible designs their own NIC chip with MIPS cores and other supporting accelerators, so the FPGA stuff obviously didn't cut it for either cost (likely), performance (FPGAs are slow compared to ASICs), or power (FPGAs are expensive and consume more power than ASICs). Even the expensive FPGA NIC went into more than 1M servers.

IMO, Microsoft is just behind the curves of AWS and Google on cloud server chips, and needs in-house designs to compete on cost.

Even Oracle invests in its own cloud CPU designs; they just did it with Ampere. Larry Ellison also had the chip design insight earliest, when Oracle acquired Sun in 2009. That didn't work out as well as he anticipated, but Ellison is often ahead of the curve, and that entails more risk.

I think bringing cloud server chip design in-house for CPUs and networking is probably the biggest threat to Intel's product groups' gross margin, and one big reason why I'm a fan of Intel getting its foundry act together. In foundry they're really one of two, maybe one of three if you count Samsung, which looks like a stretch. I think growth in the highly profitable merchant server chip market will be slower in the future, unless you're Nvidia. Very likely considerably slower. There's a loud sucking sound from enterprise data centers to the cloud.

 

[KevinK]

IMO, Microsoft is just behind the curves of AWS and Google on cloud server chips, and needs in-house designs to compete on cost.

Even Oracle invests in its own cloud CPU designs; they just did it with Ampere. Larry Ellison also had the chip design insight earliest, when Oracle acquired Sun in 2009. That didn't work out as well as he anticipated, but Ellison is often ahead of the curve, and that entails more risk.

I think bringing cloud server chip design in-house for CPUs and networking is probably the biggest threat to Intel's product groups' gross margin,

I think you nailed it. Microsoft in-house workloads are historic CPU-centric workloads, not AI, which requires entire rethinks of what a data center is. They, like Oracle, Nebius and CoreWeave are also building completely new AI data centers. So chips and optimized systems to accelerate Microsoft workloads really threaten Intel/AMD more than NVIDIA. AI data centers look more like this:

Inside the world’s most powerful AI datacenter - The Official Microsoft Blog

This week we have introduced a wave of purpose-built datacenters and infrastructure investments we are making around the world to support the global adoption of cutting-edge AI workloads and cloud services. Today in Wisconsin we introduced Fairwater, our newest US AI datacenter, the largest and...

blogs.microsoft.com

https://investors.coreweave.com/news/news-details/2025/CoreWeave-Announces-Multi-Billion-Dollar-Commitment-to-AI-Infrastructure-in-Pennsylvania/default.aspx

https://www.bloomberg.com/news/articles/2025-10-02/microsoft-neocloud-deals-a-strategy-for-ai-supply-crunch

OpenAI, Oracle, SoftBank plan five new AI data centers for $500 billion Stargate project

OpenAI, Oracle and SoftBank on Tuesday announced plans for five new artificial intelligence data centers in the United States to build out their ambitious Stargate project.

www.reuters.com

 

[blueone]

I think you nailed it. Microsoft in-house workloads are historic CPU-centric workloads, not AI, which requires entire rethinks of what at data center is. They, like Oracle, Nebius and CoreWeave are also building completely new AI data centers. So chips and optimized systems to accelerate Microsoft workloads really threaten Intel/AMD more than NVIDIA. AI data centers look more like this:

Inside the world’s most powerful AI datacenter - The Official Microsoft Blog

This week we have introduced a wave of purpose-built datacenters and infrastructure investments we are making around the world to support the global adoption of cutting-edge AI workloads and cloud services. Today in Wisconsin we introduced Fairwater, our newest US AI datacenter, the largest and...

blogs.microsoft.com

https://investors.coreweave.com/news/news-details/2025/CoreWeave-Announces-Multi-Billion-Dollar-Commitment-to-AI-Infrastructure-in-Pennsylvania/default.aspx

https://www.bloomberg.com/news/articles/2025-10-02/microsoft-neocloud-deals-a-strategy-for-ai-supply-crunch

OpenAI, Oracle, SoftBank plan five new AI data centers for $500 billion Stargate project

OpenAI, Oracle and SoftBank on Tuesday announced plans for five new artificial intelligence data centers in the United States to build out their ambitious Stargate project.

www.reuters.com

I suspected that Nvidia GPUs and NVLink are the solution for the new datacenter, because Azure's highly touted AI chip, Maia, isn't ready yet, or the software stack isn't. Whatever, that new datacenter in your link is all Nvidia. So I wonder how Maia fits in the AI strategy? Only for Azure applications, but not for world domination? More than a bit confusing to me.

 

[horace]

Why don't MSFT just build their own GPU instead of ASIC?

 

[ChrisGar]

Why don't MSFT just build their own GPU instead of ASIC?

In this context, it is the same thing.

 

[Michael Domnitei]

I visited Microsoft many years ago as an hardware engineer and actually talked with a chip designer in his office. Microsoft (like other big companies I had experience with) is a closed office culture with an atmosphere of government cold culture. I refused to work there because I sensed that Microsoft has no business in developing chips and barely made it on hardware products. Microsoft had the tablet concept developed many years before they gave it away to Apple because they were not capable to produce something to sell. But eventually Microsoft produced the surface laptop many years after they gave the technology to Apple which produce the tablets form of Microsoft idea. Microsoft has exploited Intel for decades for their own development and I remember while working at Intel having a weekly meeting in which the top logo of the slides was: "Intel is in Microsoft jail". Also, on the AI, Microsoft had tried to squeeze as much cooperation with Intel in the graphics accelerator development with the same strategy to entrap Intel in their Microsoft jail. In my opinion, having worked in chip development domain, Microsoft should stick with software unless it wants to repeat what Intel did trying to develop graphics processors and wasted billions of dollars.

 

[siliconbruh999]

I visited Microsoft many years ago as an hardware engineer and actually talked with a chip designer in his office. Microsoft (like other big companies I had experience with) is a closed office culture with an atmosphere of government cold culture. I refused to work there because I sensed that Microsoft has no business in developing chips and barely made it on hardware products. Microsoft had the tablet concept developed many years before they gave it away to Apple because they were not capable to produce something to sell. But eventually Microsoft produced the surface laptop many years after they gave the technology to Apple which produce the tablets form of Microsoft idea. Microsoft has exploited Intel for decades for their own development and I remember while working at Intel having a weekly meeting in which the top logo of the slides was: "Intel is in Microsoft jail". Also, on the AI, Microsoft had tried to squeeze as much cooperation with Intel in the graphics accelerator development with the same strategy to entrap Intel in their Microsoft jail. In my opinion, having worked in chip development domain, Microsoft should stick with software unless it wants to repeat what Intel did trying to develop graphics processors and wasted billions of dollars.

Intel is developing Graphics processor now again and this time they are doing it lot better than previous attempt. I am pretty sure WoA attempt is scaring Intel more though.

 

[Michael Domnitei]

Intel is developing Graphics processor now again and this time they are doing it lot better than previous attempt. I am pretty sure WoA attempt is scaring Intel more though.

I worked in the Graphics Accelerator (Movidius & MyriadX) group at Intel (part of the AIPG - artificial intel. platform group) and the product was very successful (the demand was beyond any expectation) so the design was expanded to other usage (ex: for datacenter applications).

 

[hist78]

I worked in the Graphics Accelerator (Movidius & MyriadX) group at Intel (part of the AIPG - artificial intel. platform group) and the product was very successful (the demand was beyond any expectation) so the design was expanded to other usage (ex: for datacenter applications).

Did Intel discontinue Movidius and Myriad products in 2022? Do you know the reason behind it?

 

[siliconbruh999]

Did Intel discontinue Movidius and Myriad products in 2022? Do you know the reason behind it?

Movidius is not discontinued their IP lives in the name of Intel NPU

 

[hist78]

Movidius is not discontinued their IP lives in the name of Intel NPU

Intel’s NPU is not a standalone product; rather, it is a component of Intel’s Core Ultra processors. It's designed to support Intel's AI PC offerings. Intel has high hopes for AI PCs, but the market has not yet shown strong demand yet.

 

[Michael Domnitei]

I don't know but I was disappointed at the time because they didn't extend Movidius which was very popular among regular people and especially universities which could enable AI applications for PhD candidates. Flir was one major user for their applications used in scanning at american ports but also there were many other companies (ex: the product known as "porch-pirate" was done with Movidius) which used Movidius but they got involved with Movidius before Intel bought the San Mateo company which started it. The silicon was developed in Ireland and the rest of development was divided between Hillsboro, Oregon and Timisoara, Romania (a very strong software team which was the main driver). Intel didn't advertised Movidius (big mistake) and expanded to MiriadX for the servers in datacenters (by adding more memory and using PCI 4.0 in 2019). Beyond that, the only thing I noticed was the involvement of Microsoft because they wanted the use of Windows environment while most users preferred Linux (Ubuntu). Also, Intel retired the original platform of Movidius and started a new one called "openvino" developed in Moscow (Russia) and some parts in China (another BS of "going global"). So, I don't know what happened with the developing team in Russia after the Ukraine war started but openvino is very advertised by Intel now and was used by all Intel AI applications (not only by Movidius & MiriadX). I retired after 2019. I assume they sold it to Microsoft (the Hillsboro manager is no longer with Intel) since there is no trace of MiriadX now.

 

[hist78]

I don't know but I was disappointed at the time because they didn't extend Movidius which was very popular among regular people and especially universities which could enable AI applications for PhD candidates. Flir was one major user for their applications used in scanning at american ports but also there were many other companies (ex: the product known as "porch-pirate" was done with Movidius) which used Movidius but they got involved with Movidius before Intel bought the San Mateo company which started it. The silicon was developed in Ireland and the rest of development was divided between Hillsboro, Oregon and Timisoara, Romania (a very strong software team which was the main driver). Intel didn't advertised Movidius (big mistake) and expanded to MiriadX for the servers in datacenters (by adding more memory and using PCI 4.0 in 2019). Beyond that, the only thing I noticed was the involvement of Microsoft because they wanted the use of Windows environment while most users preferred Linux (Ubuntu). Also, Intel retired the original platform of Movidius and started a new one called "openvino" developed in Moscow (Russia) and some parts in China (another BS of "going global"). So, I don't know what happened with the developing team in Russia after the Ukraine war started but openvino is very advertised by Intel now and was used by all Intel AI applications (not only by Movidius & MiriadX). I retired after 2019. I assume they sold it to Microsoft (the Hillsboro manager is no longer with Intel) since there is no trace of MiriadX now.

Thank you for sharing some of the history of Movidius and Myriad. I think Intel has a tendency to focus solely on x86 CPUs. Anything else is treated as ancillary to the CPUs and often ends up being canceled, folded into the CPU line, or sold to other companies. Intel has long believed that customers should follow its roadmap and technical decisions. Unfortunately, when customers have other choices, they won’t.

 

[Michael Domnitei]

Thank you for sharing some of the history of Movidius and Myriad. I think Intel has a tendency to focus solely on x86 CPUs. Anything else is treated as ancillary to the CPUs and often ends up being canceled, folded into the CPU line, or sold to other companies. Intel has long believed that customers should follow its roadmap and technical decisions. Unfortunately, when customers have other choices, they won’t.

If I remember correctly, Movidius had at least 20 Arm processors on its die.

 

[Xebec]

Intel is developing Graphics processor now again and this time they are doing it lot better than previous attempt. I am pretty sure WoA attempt is scaring Intel more though.

I always saw WoA as a desperate attempt to stave off the loss of Windows users to devices like the iPad