Last I covered the consumer story — DLSS 5, the CUDA flywheel, and the economics that explain why $1 trillion in orders makes sense. this is the hardware. And this is where I want to take my time, because the Vera Rubin announcement is genuinely more sophisticated than most coverage is giving it credit for.
It’s not just a faster GPU. It’s a complete rethinking of what an AI computing system is, how it’s structured, and how the different chips inside it are supposed to work together. Let me explain it properly.
The Era of Holding Up a Chip Is Over
For years, Huang’s signature GTC move was producing a GPU wafer from somewhere — his jacket, a display case, a presentation table — and holding it up. One chip. Hero object. The crowd responds. It was great theater.
He can’t do that anymore. Not because the chips got worse, but because the chip is no longer the unit of compute.
“The unit of compute today is the rack,” he said. And then he unveiled the Vera Rubin NVL72 — a single liquid-cooled rack containing 72 Rubin GPUs, 36 custom Vera CPUs, and 1.3 million individual components. The installation time for one of these racks has dropped from two days to two hours, and it runs on 45°C hot water cooling that shifts the thermal management burden away from the data center and into the rack itself.
Peak output: 700 million tokens per second from a single rack.
The previous generation delivered 22 million tokens per second from a 1GW data center. Vera Rubin, in the same power envelope, delivers 700 million. That is a 350x improvement in two years — versus roughly 1.5x from Moore’s Law over the same period. Huang acknowledged the number sounds unbelievable and took his time justifying it. The justification is the extreme co-design story — hardware and software built together from the ground up, optimizing every stage of the inference pipeline.
And it’s not vaporware. Satya Nadella confirmed publicly that the first Vera Rubin rack is already running inside Microsoft Azure. It shipped.
For anyone shopping gaming desktops at Newegg or considering a high-end GPU upgrade: Vera Rubin is the architecture that the next consumer GeForce generation will draw from. The current RTX 50 Series is Blackwell — excellent hardware, available now. The Rubin-derived consumer cards come in 2027. Knowing that timing is useful for planning.
The Vera CPU: Built From Scratch for AI
Alongside the Rubin GPU, NVIDIA has introduced a completely new custom CPU called Vera — and it’s worth understanding why this exists and what problem it solves.
Traditional server CPUs aren’t built for the specific access patterns that AI inference demands. When a model is running, it needs to constantly pull parameters and KV cache data from memory at enormous bandwidth. Existing CPU architectures have latency and bandwidth characteristics optimized for a different class of workload.
Vera was built from scratch specifically for this. It’s the only data center CPU in the world using LPDDR5 memory — a design choice that gives it exceptional performance per watt for AI inference workloads, particularly for the tool-calling operations that AI agents perform constantly. 2x the single-thread performance of the previous Grace chip. 1.5TB memory, 1.2 TB/s bandwidth.
Huang’s line: “AI will not tolerate slow computers.” Vera is the CPU that ensures the GPUs never have to wait.
Groq: The Solution to the “Last Mile” Inference Problem
This is the most technically interesting part of the hardware story, and it got significantly less coverage than it deserved.
Here’s the underlying problem: large language models have two distinct computational phases — prefill (processing the input context, doing the “thinking”) and decode (generating the output tokens one by one). Prefill is compute-intensive and parallelizable — GPUs are excellent at it. Decode is sequential and latency-sensitive. Every token has to be generated before the next one can start. At the extreme end — think real-time code generation or live agentic reasoning — the latency of decode becomes the bottleneck, and GPUs, with their deep memory hierarchies and high-bandwidth but relatively high-latency access patterns, aren’t optimized for it.
Groq built a chip called the LPU (Language Processing Unit) that is. It has 500MB of on-chip SRAM — tiny compared to the 288GB in a Rubin chip, but directly accessible at extremely low latency. It compiles the decode path statically at model load time, eliminating the scheduling overhead that makes GPUs less efficient for this phase. The result: extreme token generation speed at low latency.
The limitation: 500MB of SRAM isn’t enough to run a full large model’s parameters and KV cache. You can’t use Groq standalone for enterprise-scale inference.
Dynamo solves this. NVIDIA’s inference orchestration software disaggregates the inference pipeline — sending prefill and KV cache work to Vera Rubin, sending the feed-forward decode work to Groq. The two systems run in parallel over Ethernet with a special protocol that cuts latency roughly in half. Together, they open up performance tiers that neither could reach independently. The result: 35x improvement in tokens-per-watt over GPU-only configurations, unlocking the high-value $45-$150/million token pricing tiers Huang described in Part 1.
Groq LP30 is manufactured by Samsung, in production now, expected to ship Q3 2026.
Huang’s practical recommendation on how to allocate infrastructure: “If your workload is primarily high throughput, use 100% Vera Rubin. If you have significant high-value code generation or agentic token needs, allocate roughly 25% of your data center capacity to Groq.”
For consumers, the implication is downstream but real: the latency and efficiency improvements being built into this disaggregated inference architecture are what will make AI features in consumer products — game NPCs, copilot features, creative tools — feel instant rather than sluggish. Infrastructure improvements filter into products. They always do.
The Roadmap: Unusually Clear, Unusually Useful
Huang closed the hardware segment with a four-generation roadmap that was more specific than NVIDIA typically shares.
Now — Blackwell (RTX 50 Series): In production. The consumer GPU you can buy today at Newegg. Receiving DLSS 5 as a driver update later this year. If you’re upgrading from RTX 30 Series or earlier, this is your generation. Browse RTX 50 Series desktop GPUs and gaming laptops — in stock now.
2026 — Vera Rubin (data center): Shipping now. Consumer derivatives in 2027.
2027 — Vera Rubin Ultra + Kyber rack: New Rubin Ultra GPU with LP35 chip (first to integrate NVFP4 in this generation), vertical-insertion Kyber rack architecture scaling to 144 GPUs in a single NVLink domain. Consumer-facing hardware late 2027.
2028 — Feynman: New GPU (LP40, co-developed with the Groq team, NVFP4 integrated), new CPU called Rosa (Rosalyn), BlueField 5, CX 10, CPO + copper dual-path Kyber rack. The generation Huang is most excited to not say too much about yet.
If you’re on RTX 40 Series, the honest advice: Blackwell is a solid upgrade from Ampere, but a more moderate step from Ada Lovelace. Rubin-derived consumer hardware in late 2027 will likely be the more significant jump. Sit tight if you can.
DSX and the AI Factory Digital Twin
One more announcement worth calling out before we move to software: NVIDIA DSX — a digital twin platform for designing and operating gigawatt-scale AI factories.
The problem it solves is real: a modern AI data center involves hundreds of vendors — GPU makers, switch makers, cooling companies, power suppliers, rack designers — who historically never collaborated during the design phase. They’d each ship their products and discover compatibility and efficiency problems only after installation.
DSX puts all of them in a shared virtual environment — mechanical, thermal, electrical, and networking simulation all in one place, connected to actual grid data for cooperative energy scheduling. Conservative estimate on efficiency gains: 2x. At gigawatt scale, that’s meaningful money.
It also hints at NVIDIA’s broader ambition — Huang described Omniverse as a platform that will “carry all scales of digital twins,” and DSX is its first major data center application. The long-term vision is a digital twin of the entire AI infrastructure buildout — simulated before it’s built, optimized before it runs.
Next— Part 3: OpenClaw, the SaaS-to-GaaS transition, NVIDIA going to space, self-driving Ubers in 28 cities, and the robot snowman. Newegg Insider.
Related Posts
- Sales Tax Holiday 2026: Save on PC Components, Peripherals, Laptops, Desktops, and Workstations at Newegg
- Apple Intelligence Features at WWDC 2026: The New Siri, Photo Editing, and What Your Device Needs
- Apple Parental Controls at WWDC 2026: The New Child Account Tools Explained
- iOS 27 Performance Boost: Every Speed and Design Change Apple Showed at WWDC 2026
- Intel’s Fastest Gaming Desktop CPUs Are Almost Here: A Complete Look at the Core Ultra 270K/250K/250KF Plus Arrow Lake Refresh