Microsoft's Maia 300 puts its own silicon at the center of AI inference
Microsoft introduced Maia 300 on Monday, the third generation of its in-house AI accelerator, and said it will move into production in 2027 with deployments across its datacenters at multi-gigawatt scale. The company presents it as a complete system — silicon, software, rack infrastructure, networking and datacenters co-designed as one platform — rather than a chip sold on its own. The architecture also breaks from Nvidia's formula of one large compute die surrounded by high-bandwidth memory: Maia 300 is built from multiple chiplets, with compute elements made on TSMC's N2P 2-nanometer process and cheaper 3-nanometer chiplets handling data movement. Each accelerator carries 288 GB of HBM4 memory across eight stacks, 18 TB/s of bandwidth, 13.8 PFLOPS at FP4 and 6.9 PFLOPS at FP8, within a 950-watt thermal design power. Racks hold 96 accelerators, twice the Maia 200 configuration, and draw about 167 kilowatts, versus roughly 142 kilowatts for Nvidia's latest GB300 racks with 72 GPUs.
Scott Guthrie, executive vice president of Microsoft's cloud group, wrote that Maia will become "the primary AI inference system for Microsoft products and services," with the Azure Maia family scaled to multi-gigawatt deployments starting in 2027. His headline figures: 40% more tokens per megawatt than "leading GPU-based systems," up to 7x the rack-scale token throughput of Maia 200 in a full 96-accelerator configuration, and more than 2.5 times the performance per watt of the previous generation. Reuters noted that Microsoft did not name the GPU systems used as the comparison. Microsoft says the chip is designed for the workloads it sees growing fastest — long-context reasoning, latency-sensitive conversation and sustained, multi-step agentic tasks — using a software-defined dataflow architecture with AI-assisted kernel optimization to keep data close to the compute that needs it.
The install base is not starting from zero. Maia 200, launched in January, already runs Microsoft's own MAI models, OpenAI models delivered through Azure and open-source models, and it supports production workloads including Microsoft 365 Copilot. Microsoft says it delivers 1.7 times more tokens per watt than GPU-based systems for AI features in Teams, Copilot and Cowork. The rollout so far, however, has been modest: The Information reported in August that as of this summer Maia 200 was running in only two U.S. data centers. The first Maia was announced in 2023, and the second arrived in January after delays.
Microsoft expects capital expenditures to exceed $175 billion this calendar year, and cloud infrastructure providers are facing escalating expenses and power limits. Rani Borkar, president of Azure hardware systems and infrastructure, told Reuters the goal is "delivering more intelligence for every watt, every dollar, and every byte of memory." GeekWire called the announcement Microsoft's "biggest bet yet" on its own silicon and reported that the shift could save the company billions of dollars while reducing its dependence on Nvidia.
The change is scoped to inference. Alistair Speirs, a Microsoft general manager, told CNBC that the company continues to partner with Nvidia and AMD on training models, and that Nvidia GPUs remain deployed across its existing datacenters. Nvidia chief Jensen Huang appeared at a Microsoft event last week for a Surface laptop containing an Nvidia processor. Evercore ISI estimates that Maia will represent about 10% of Microsoft's spending on AI chips by the first quarter of 2027, and the firm recommends buying Microsoft shares. The stakes are asymmetric: media coverage of the announcement pointed out that Nvidia regularly generates upwards of 50% of its quarterly revenue from hyperscalers, and Amazon and Google now rent their own custom chips to outside customers. Azure customers, meanwhile, cannot rent Maia by the hour, so the swap happens invisibly behind Microsoft's own products. Speirs declined to give a timeline, saying the company keeps "a little bit of optionality, because this is an industry that's changing very fast."
Reuters reported that Microsoft plans to deploy potentially more than a million of its custom chips next year to back internal AI services, with Maia 300 production starting in 2027. Earlier reporting cited by Quartz said Microsoft was negotiating with TSMC for capacity covering upward of 300,000 Maia 300 units for delivery in 2027, while eyeing eventual production above one million units. Rival programs are moving at similar speed — Google, Amazon and OpenAI are all expanding their own chip efforts — but Microsoft is the one tying its consumer and enterprise AI stack, from Copilot to Foundry, to silicon it designs itself. Whether the third generation scales faster than the first two is the open question.