Across the global hyperscale landscape, cloud AI chips news today is defined by a structural shift: while merchant GPUs from NVIDIA and AMD remain the primary workhorses for multi-tenant cloud clusters, the three largest cloud computing providers — Amazon Web Services (AWS), Google Cloud and Microsoft Azure — are scaling first-party custom AI silicon at unprecedented volume. By co-designing custom ASICs alongside proprietary scale-up fabrics, rack power shelves and liquid-cooling manifolds, hyperscalers eliminate merchant gross-margin premiums and tailor silicon specifically to internal foundation models and high-volume inference workloads.
AWS and Amazon AI chips: Trainium2, Trainium3 and Inferentia UltraServers
In AWS AI chips news today (and broader Amazon AI chips news today), Annapurna Labs — Amazon’s in-house silicon division — has moved from standalone PCIe inference cards to tightly coupled, rack-scale UltraServers:
- Trainium2 and Trn2 UltraServers: Built on advanced 5nm-class foundry silicon with high-capacity HBM3/HBM3E memory, Trainium2 delivers roughly 4× the training compute and 3× the memory capacity of first-generation Trainium. In Trn2 UltraServer configurations, 64 Trainium2 chips are interconnected via low-latency NeuronLink scale-up fabrics within a high-density rack footprint to serve trillion-parameter foundation models.
- Trainium3 on 3nm silicon: Moving to a 3nm process node, Trainium3 targets a further 2× compute jump and ~40% energy-efficiency improvement inside next-generation UltraClusters, paired with Amazon’s Elastic Fabric Adapter (EFA) scale-out networking.
- AWS Inferentia2 for high-concurrency serving: For cost-sensitive production inference across Amazon Bedrock and Alexa services, Inferentia2 pairs NeuronCore-v2 engines with HBM memory and native FP8/BF16 quantization support through the AWS Neuron SDK (integrated with PyTorch and vLLM).
Google AI chips: TPU v6e (Trillium) and Ironwood 3D optical pods
No company has iterated custom data centre accelerators longer than Google. In Google AI chips news today, seventh-generation Tensor Processing Units (TPUs) anchor both Gemini frontier training and Google Cloud Vertex AI inference:
- TPU v6e (Trillium): Delivers a 4.7× peak compute increase per chip over TPU v5e while doubling HBM capacity and inter-chip interconnect (ICI) bandwidth, achieving over 67% better energy efficiency per watt.
- Ironwood (7th-Gen Inference TPU): Engineered specifically for the reasoning and long-context inference era, Ironwood scales up to 9,216 liquid-cooled chips per pod linked via a high-speed Inter-Chip Interconnect (ICI) mesh and optical circuit switches (OCS), delivering 4,614 TFLOPs of peak FP8 compute per chip alongside 192 GB of HBM3E memory at 7.2+ TB/s bandwidth.
Microsoft AI chips: Azure Maia 100, Maia 200 and Sidekick liquid cooling
In Microsoft AI chips news today, Azure’s custom silicon roadmap combines in-house accelerator design with purpose-built thermal infrastructure so new chips can slot directly into existing Azure data halls:
- Azure Maia 100 & Sidekick rack cooling: Fabricated on TSMC’s 5nm process with 105 billion transistors and 64 GB of HBM2E memory, Maia 100 was deployed with custom closed-loop liquid-to-air Sidekick heat-exchanger racks paired with Azure Cobalt 100 Arm server CPUs.
- Maia 200 & next-generation HBM3E/HBM4 transition: Microsoft’s follow-on Maia generations move to 3nm-class lithography and high-stack HBM3E memory, utilizing standardized Ultra Ethernet / RoCEv2 backend fabrics to reduce token generation cost across Copilot and Azure OpenAI Service workloads.
| Cloud Provider & Chip Family | Primary Workload Focus | Scale-Up Interconnect & Rack Architecture | Software Compiler Stack |
|---|---|---|---|
| AWS (Amazon) Trainium2 / Trainium3 & Inferentia2 | Frontier LLM training & Bedrock high-throughput inference | 64-chip NeuronLink Trn2 UltraServer + 800G EFA scale-out | AWS Neuron SDK (PyTorch, XLA, vLLM, Triton) |
| Google Cloud TPU v6e (Trillium) & Ironwood | Gemini training, reasoning models & large-scale pod inference | 2D/3D Torus ICI + Optical Circuit Switching (up to 9,216 chips) | OpenXLA, JAX, PyTorch/XLA, Pathways |
| Microsoft Azure Maia 100 / Maia 200 | Azure OpenAI & Copilot first-party cloud inference | Custom Ethernet rack fabric + Sidekick direct-to-chip liquid cooling | Maia SDK, Triton, ONNX Runtime & PyTorch |
What cloud computing AI chips mean for enterprise infrastructure buyers
- Evaluate portable software stacks early: Teams building on OpenAI Triton, standard PyTorch 2.x (`torch.compile`), JAX/OpenXLA or vLLM can migrate inference workloads onto AWS Trainium or Google TPU instances for 30% to 50% lower cost per million tokens compared to scarce merchant GPU instances.
- Match instance topology to model parallelism: When serving 70B+ or 405B+ models, prioritize multi-chip domains with dedicated chip-to-chip fabrics (such as Trn2 UltraServers or TPU pods) rather than standard PCIe-attached cloud VMs.
For deeper facility and rack engineering context, visit our Data Centres coverage hub, review our Broadcom and Anthropic TPU infrastructure analysis, or consult our guide on evaluating AI accelerator software stacks.