Microsoft has announced plans to scale its Azure Maia AI accelerator family to multi-gigawatt deployments beginning in 2027, with the new Maia 300 chip positioned as the company's primary inference system for its products and services. Maia 200 is already serving production workloads including Microsoft Copilot, delivering up to 1.7 times more tokens per watt than leading GPU infrastructure. The announcement details significant architectural improvements designed to handle emerging AI workloads that require extended context windows and sustained multi-step processing.
Architectural focus on inference efficiency
Maia 300 introduces several technical innovations aimed at reducing the energy and computational costs of running large AI models. The chip uses a multi-chiplet architecture with compute chiplets fabricated on TSMC's N2P process node. Eight HBM4 memory stacks provide 18 terabytes per second of peak bandwidth within a 950-watt thermal envelope, delivering industry-leading memory bandwidth per watt according to Microsoft.
The design employs software-defined dataflow architecture, which treats data movement across compute, memory, and networking as a unified system. The chip also supports reduced-precision formats including enhanced MXFP4, which decreases memory requirements for model parameters and attention cache data.
At the rack level, Maia 300 systems integrate 96 accelerators within a 167-kilowatt power budget. A full 96-accelerator configuration delivers up to 7 times the rack-scale token throughput of its predecessor. Microsoft projects Maia 300 will provide 40 percent more tokens per megawatt than leading GPU-based systems.
This infrastructure expansion addresses a fundamental constraint on AI deployment: the energy and hardware costs of serving increasingly capable models at scale. As enterprises move from simple chat interfaces to AI systems that can reason through extended tasks and use external tools, computational demands on datacenters grow substantially. The multi-gigawatt deployment commitment indicates the scale at which Microsoft anticipates AI inference will operate within its cloud infrastructure.
For additional technical details, Microsoft has published a deep dive into the Maia 300 architecture.