NVIDIA Vera Rubin Platform Ramping into Full Production | Built for the Era of Agents
Agentic AI requires systems that support observation, reasoning, planning, tool use, large working and long‑term memory, and dynamic sub‑agent spawning; conventional single‑chip or single‑rack designs are insufficient. NVIDIA Vera Rubin is a multi‑rack pod‑scale supercomputer built for this agentic era: heterogeneous, liquid‑cooled racks that combine NVL72 superchips for high‑throughput token generation, Groq 3 LPX for ultra‑low latency, BlueField‑4 DPUs for memory/storage/security acceleration, and Spectrum‑X photonic switching. The build is evidenced by the production rollout: seven new 3 nm chips, CoWoS packaging, HBM4 from multiple vendors, a 6‑trillion‑transistor compute board, MGX third‑generation racks with 1.3M components, and five connected rack‑scale systems now in full production. Outcome: a purpose‑built pod‑scale platform that trades extreme power, cooling, and supply‑chain complexity for agentic capabilities and deployment resilience (hot‑swappable NVLink switches, cableless interconnects), with the practical limitation of very large manufacturing and operations scale.
Key technical points
- Problem context: agentic AI workloads require systems that handle observation, reasoning, planning, tool invocation, large context (working and long‑term memory), and dynamic sub‑agent creation—demands beyond prior single‑rack designs.
- System architecture: Vera Rubin is a pod‑scale supercomputer composed of five connected rack‑scale systems (heterogeneous racks tuned to different roles) designed to run agentic workloads.
- Heterogeneous compute tiers: NVL72 (MVL‑72) superchips focused on prompt/context understanding, reasoning, planning, and highest token throughput; Groq 3 LPX trays focused on ultra‑low latency token generation.
- Compute hardware details: seven new 3 nm chips, CoWoS‑R and CoWoS‑L packaging, HBM4 memory supplied by Micron, SK Hynix, and Samsung; the Vera compute board contains ~6 trillion transistors and >18,000 components.
- Rack design and connectivity: third‑generation MGX rack with 1.3 million components, modular compute trays, a new PCB midplane design, 18 compute trays per rack, nine hot‑swappable NVLink switch trays, and cableless mating of Superchips, ConnectX‑9 SuperNICs, and BlueField‑4 DPUs for resiliency.
- Power and cooling: liquid‑cooled bus bars carrying over 5,000 amps per system (scale compared to 20 electric cars at full acceleration), plus new high‑efficiency manifolds—indicating major facility power and liquid‑cooling requirements.
- Orchestration and control plane: a dedicated Vera CPU rack containing 256 liquid‑cooled CPUs per rack to orchestrate models, shuffle memory, and launch tools—separating control/orchestration from token generation compute.
- Memory and storage acceleration: BlueField‑4 STX DPUs accelerate storage processing and act as the memory/storage/security interconnect (described as where AI keeps its memory), providing in‑silicon security functions.
- Networking and photonics: NVIDIA Spectrum‑X Ethernet switch is the first with 200 Gbps co‑packaged optics, using TSMC CoWoS, chip‑scale packaging, and indium phosphide laser diodes—targeted at extremely high bandwidth/low‑latency interconnects.
- Latency vs throughput trade‑off: MVL‑72 (NVL72) optimized for highest throughput token generation, while Groq 3 LPX is optimized for lowest latency token generation—implying heterogeneous placement of workloads by latency/throughput needs.
- Resiliency and serviceability design choices: hot‑swappable NVLink switch trays and cableless mating aim to improve resiliency and maintenance at AI factory scale.
- Scale of manufacturing and supply chain: production involves ~150 supply‑chain partners across Taiwan, millions of square feet of factory floor, and hundreds of sites—characterizing extreme co‑design and complex manufacturing orchestration.
- Deployment evidence: Vera Rubin has moved into full production with engineering racks operational at partners (Microsoft, Dell, Corweave) and mass manufacturing partners (Foxconn, Quanta) building Groq LPX trays.
- Component and bandwidth numbers called out: Groq 3 LPX deployment example—256 Groq 3 LPUs across 16 trays delivering ~40 petabytes/second of SRAM bandwidth in that tier.
- Co‑design emphasis: the system and its manufacturing are described as an integrated effort with Taiwan partners from chip process through packaging, boards, racks, and data‑center deployment—an explicit example of extreme co‑design.
For engineers designing and operating agentic AI systems, Vera Rubin demonstrates several concrete implications: architect workloads against heterogeneous compute tiers (throughput‑optimized NVL72 vs latency‑optimized Groq LPX) and separate orchestration/control plane (256‑CPU Vera CPU rack) to manage memory shuffling and tool launching. Systems must integrate DPUs (BlueField‑4) to offload storage processing and enforce in‑silicon security, and use high‑bandwidth photonic switching (Spectrum‑X) to meet interconnect demands. Physical infrastructure planning must account for extreme power (5,000+ amps per liquid‑cooled bus), dense liquid cooling, hot‑swappable NVLink switch maintenance, and cableless interconnect topologies to achieve resiliency at scale. Production readiness requires supplier coordination, CoWoS packaging and HBM4 integration, and acceptance of the manufacturing complexity inherent in multi‑rack pod designs. Finally, expect trade‑offs between throughput and latency across compute tiers and design placement/orchestration policies accordingly.
For Japanese engineering teams evaluating or integrating pod‑scale agentic AI: anticipate requirements for datacenter upgrades (high‑capacity liquid cooling, electrical infrastructure sized for thousands of amps), and include mechanical/design work for hot‑swap serviceability and cableless midplane connections. Procurement and integration plans should factor in multi‑vendor supply chains (HBM4 from multiple memory vendors, packaging from TSMC CoWoS processes) and collaborate early with OEM partners (vendor examples in the transcript include Microsoft, Dell, Foxconn, Quanta) for engineering rack validation. Architect applications to leverage heterogeneous tiers—route latency‑sensitive workloads to Groq LPX tiers and throughput/batch workloads to MVL‑72—while using a dedicated CPU orchestration tier to manage memory and tool execution. Finally, recognize the importance of co‑design with supply partners (as NVIDIA did with Taiwan) when planning large deployments to ensure manufacturability, serviceability, and operational resiliency.