MLPerf Training v6.0, developed by MLCommons (MLPerf Benchmarks), introduced DeepSeek-V3 671B and GPT-OSS-20B mixture-of-experts pretraining workloads, and CoreWeave used the opportunity to demonstrate a new scale-out benchmark result. CoreWeave trained DeepSeek-V3 671B in 2.02 minutes on 8,192 GB300 NVL72 GPUs, which was the largest GB300 cluster submitted in the round. This time-to-train figure, along with NVIDIA Blackwell submissions across 19 organizations, provides a detailed view of how rack-level NVLink, scale-out networking, and production cloud tooling affect large-model training.
Blackwell Ultra Rack-Level Gains
The GB300 NVL72 rack connects 72 Blackwell Ultra GPUs and 36 Grace CPUs through fifth-generation NVLink Switches, forming a unified compute and memory pool. NVIDIA states that Blackwell Ultra gains include NVFP4 compute density, more memory, and a higher power ceiling. Those gains translated into a 1.6x training speedup for GB300 NVL72 over GB200 NVL72 at the same scale. NVIDIA itself submitted a 5,120 GPU run on Llama 3.1 405B using GB200 NVL72, while Microsoft Azure reached the Llama 3.1 405B quality target in 7.07 minutes at 8,192 GPUs. Across the round, NVIDIA partners scaled to 8,192 Blackwell GPUs in diverse cloud data centers, including Google Cloud, Cohere, Thinking Machines Lab, and Dell Technologies. NVIDIA screens GPUs across more than 30 manufacturing test stages before deployment, and its NVRx tool resumes training from a recent checkpoint rather than restarting an entire job when hardware faults occur. For deeper context on how these submissions compare with prior rounds, see the AI chip benchmarks hub.
Networking for MoE Expert Parallelism
DeepSeek-V3 671B is a mixture-of-experts model that serves as the base for DeepSeek-R1 reasoning. At 8,192 GPUs, expert parallelism creates low-entropy bursty flows that reduce equal-cost multipath bandwidth. NVIDIA's ConnectX SuperNIC handles out-of-order packet delivery, and Spectrum-X Ethernet congestion control detects incast early and paces senders before buffer overflow. NVIDIA offers Quantum InfiniBand and Spectrum-X Ethernet as scale-out networking platforms. CoreWeave uses rail-aware networking to balance traffic and prevent fabric hotspots, and its SUNK scheduler co-locates expert-parallel groups within the same NVL72 domain. CoreWeave was the only submitter scaling GB300 beyond 2,048 GPUs on DeepSeek-V3, with additional results of 3.09 minutes at 4,096 GPUs and 5.54 minutes at 2,048 GPUs. The company also trained Llama-3.1-405B in 9.77 minutes on 4,096 GB300 GPUs, using 20 percent fewer GPUs than larger GB200 deployments. More details are in NVIDIA's technical blog.
Production Cloud Engineering and Software Optimizations
CoreWeave Mission Control performs health checks on GB300 rack systems before and during jobs, and the SUNK scheduler's expert-parallel co-location depends on the same rail-aware network design. CoreWeave holds the top Platinum ranking in SemiAnalysis ClusterMAX 1.0 and 2.0, and it is cited as a leading provider for Kimi K2.6 inference speed and price-performance. Midjourney trained its v8 image model on Blackwell and is now scaling a Blackwell Ultra fleet on CoreWeave. Chen Goldberg, Executive Vice President of Product and Engineering at CoreWeave, said, "Training DeepSeek-V3 in two minutes on the largest GB300 cluster reflects years of metal-to-model engineering investment." He added, "These results came from the same infrastructure our customers run in production today, not a benchmark-only setup. That's what an AI-native cloud is built to do."
Software optimizations include an MXFP8 attention recipe for MoE training that replaces 16-bit attention precision, paged stashing that manages device memory without host involvement for CUDA graph compatibility, and the CuTe DSL that enabled kernel fusions keeping data in registers and avoiding global memory round-trips. Those techniques produced a 93 percent end-to-end speedup on GPT-OSS-20B. CoreWeave trained GPT-OSS-20B in 26.98 minutes on a 64 GPU HGX B200 cluster and Llama-3.1-8B in 16.5 minutes on the same 64 GPU cluster. Brendan Burke, Research Director at Futurum Research, said, "The gap between benchmark performance and production reality remains one of the most persistent challenges in AI infrastructure." He also noted, "CoreWeave's MLPerf Training v6.0 results, particularly training DeepSeek-V3 in two minutes on the largest GB300 cluster in the benchmark, demonstrate that full stack AI expertise compounds real-world performance gains as new hardware arrives."
Other submissions on GB300 NVL72 included GPT-OSS 20B in 7.43 minutes at 512 GPUs, Llama 3.1 8B in 4.46 minutes at 1,024 GPUs, Llama 2 70B LoRA in 0.4 minutes at 512 GPUs, FLUX.1 in 17.1 minutes at 512 GPUs, and DLRM-dcnv2 in 0.67 minutes at 64 GPUs. The MLPerf Training v6.0 results were retrieved from the MLCommons website on June 16, 2026. The software stack work in CuTe DSL and MXFP8 attention demonstrates why evaluating the AI accelerator software stack matters as much as raw FLOPs for production training clusters. CoreWeave's full submission is documented in its MLPerf Training v6.0 announcement.