Microsoft announced its Maia 300 inference accelerator, increasing high-bandwidth memory bandwidth from Maia 200’s 7.1 TB/s to 18 TB/s.

Microsoft announced Maia 300 as the next generation of its custom AI inference platform. Its architecture description positions it as a single platform for cloud-scale agentic inference, rather than a general-purpose server CPU announcement.

HPCwire reports that high-bandwidth memory (HBM) bandwidth increases from 7.1 terabytes per second on Maia 200 to 18 terabytes per second on Maia 300. The publication characterizes this as a 2.5-fold bandwidth increase; the figures measure memory data-transfer bandwidth, not memory capacity or model output speed.

The article connects the bandwidth upgrade with increased ability to move data for inference and lower per-token energy cost. Separately, a supplied Telegram report states that Maia 300 is intended to deliver 40% more tokens per megawatt, but it supplies neither the comparison baseline nor test conditions, so that figure remains a reported product claim rather than an independently documented benchmark.