What percentage of the data center market will be AI/ML

[Arthur Hanson]

How long will it be until AI/ML will dominate the data center using AI/ML optimized software dominate? Will datacenters be the optimum way to utilize AI/ML resources? If not, what will the alternative be?

 

[KevinK]

That's an challenging question, especially since many software applications run in data centers appear to be moving to a mixed model with CPUs handing the code data and legacy app functionality, but with agentic/AI front-ends. But it also seem like GPUs are increasingly used for workloads that look “traditional CPUs,” especially read-heavy SQL analytics and vector-ish operations, but they are not replacing CPUs for OLTP or generic app logic.
GPUs are creeping in on
• Analytical SQL (OLAP): GPU-accelerated databases (HeavyDB, Kinetica, SQream, Brytlyt, Microsoft GPU SQL prototypes, etc.) show roughly 5–25× speedups on scan/aggregate/join-heavy queries versus CPU engines on similar cost hardware.
• GPU layers on top of CPU DBs: Systems like SiriusDB and others bolt a GPU execution engine under DuckDB/Postgres-style SQL, offloading big parallel scans, joins, and aggregates to GPUs while leaving the rest to the CPU engine.
• Vector/AI features inside RDBMS: Oracle Database 23ai, SQL Server 2025 previews, and cloud warehouses now use GPUs for vector embedding generation, vector index build, and some search paths; the rest of the SQL engine stays CPU.
These are all “SQL workloads,” but specifically analytics that are highly parallel—column scans, group-bys, hash joins, vector ops.
CPUs still dominate
• OLTP / transactional SQL: High‑churn, row‑oriented workloads (banking, ERP, web apps) still run on CPU-only engines; the fine-grained locking, branching, and latency sensitivity don’t map well to GPUs.
• General application logic: Orchestration, query planning, connection handling, stored procedures, business rules are CPU-heavy and stay that way. GPUs just act as accelerators for tight numeric kernels.