NVIDIA Vera: Rebuilding the CPU for Agentic AI

Source: Jonathon Evans and Polychronis Xekalakis, “NVIDIA Vera CPU,” Hot Chips 2026. Performance figures are NVIDIA claims, and several results were based on preproduction or unofficial testing.

NVIDIA’s Vera CPU is a server processor designed specifically for agentic AI, where language models repeatedly observe, reason, invoke tools, access memory, and act. These workflows depend on GPUs for model inference, but they also generate substantial CPU work: browser execution, orchestration, data preparation, security enforcement, storage access, networking, and tool calls. NVIDIA’s argument is that conventional cloud CPUs emphasize aggregate core count, whereas agentic systems require strong, predictable single-thread performance at rack scale.

Vera is built around 88 custom Olympus cores supporting 176 threads. Unlike a conventional chiplet processor, it uses a monolithic compute die, placing cores, cache, memory controllers, coherency infrastructure, and I/O in one tightly integrated design. This reduces the latency and unpredictability associated with crossing chiplet boundaries. The processor includes 164 MB of L3 cache and NVIDIA’s second-generation Scalable Coherency Fabric, providing 3.4 TB/s of bisection bandwidth so cores can collaborate on large datasets without creating internal bottlenecks.

The Olympus core is a wide out-of-order design. Its front end can fetch 16 instructions and decode ten per cycle, supported by a large branch predictor, a 64 KB instruction cache, and a 96 KB data cache. Each core also has a private 2 MB L2 cache. A ten-micro-operation allocation and commit path feeds multiple integer, vector, branch, load, and store units. These resources are intended to accelerate irregular CPU workloads such as Python execution, compilation, code analysis, databases, event simulation, and browser automation.

A notable feature is spatial multithreading. Traditional simultaneous multithreading allows two threads to compete dynamically for shared execution resources, which can increase total throughput but produce “noisy neighbor” effects. Vera instead partitions key resources between threads. This sacrifices some opportunistic sharing in exchange for deterministic performance and isolation. NVIDIA reports that loaded per-core performance remains near 100 percent, compared with 67 percent for a traditional CPU in its example. Internal preproduction SPEC CPU comparisons show gains ranging from approximately 1.2 to 1.8 times, with the largest improvements appearing in Python, GCC, LLVM, and code-analysis workloads.

Power and memory provisioning are equally important. Vera uses data-center-class LPDDR5X-9600 rather than conventional DDR5 DIMMs. The processor supports up to 1.5 TB of SOCAMM LPDDR5X memory with 1.2 TB/s of bandwidth. NVIDIA claims roughly five times the memory-bandwidth-per-watt of several DDR5 configurations. This choice supplies every core with sufficient bandwidth while reducing the power consumed by memory movement, helping the processor sustain frequency as utilization rises.

Vera also emphasizes external connectivity. Its aggregate off-die bandwidth reaches 4.3 TB/s. A coherent NVLink-C2C interface supplies 1.8 TB/s between the CPU and Rubin GPUs, while PCIe Gen 6 contributes 256 GB/s and supports CXL 3.1. These interfaces are intended to keep GPUs, memory, storage, and networking continuously supplied with work. In NVIDIA’s full Vera Rubin NVL72 system, 36 Vera CPUs support 72 Rubin GPUs.

Security extends across this CPU-GPU complex. Vera provides confidential-computing trusted execution environments, memory encryption, protected I/O, and security checks across coherent CPU-GPU communication. Integration with BlueField data-processing units and NVIDIA’s DOCA software is intended to protect data and workloads across an entire rack rather than only within an isolated processor.

Why does this matter? Agentic AI changes the system bottleneck. Generating model tokens is only one stage; an agent must also interpret results, run software, retrieve context, communicate with services, and enforce governance. If CPU latency delays these steps, expensive GPUs sit idle and user interaction slows. Strong single-thread performance therefore becomes as important as aggregate throughput.

Bottom line: Vera represents NVIDIA’s attempt to co-design the entire AI factory rather than optimize the GPU alone. Its custom cores, deterministic threading, high-bandwidth low-power memory, coherent GPU links, and rack-level security treat the CPU as an active component of AI performance. The larger significance is architectural: future AI systems may be judged not by the speed of any individual chip, but by how efficiently CPUs, GPUs, memory, networking, storage, and security operate as one coordinated machine.

Also Read: