Header image generated by AI

Authored by | Tian Yu

Reviewed by | Shen Xiao

On the evening of October 12, Microsoft's Chairman and CEO, Satya Nadella, unveiled the company's latest strategy for its proprietary chips on social media.

By 2027, Microsoft envisions a widespread deployment of its Azure Maia 300 chip.

On the same day, Microsoft's stock surged by 1.14%, reaching $541.15, and its market capitalization rebounded to $4 trillion.

This announcement has sparked widespread discussion.

Optimists view Microsoft's move as a direct challenge to NVIDIA, which has established itself as a dominant force in AI computing infrastructure with its GPU and CUDA ecosystem, emerging as a key player in the AI era. OpenAI's training of its GPT series models, Microsoft's operation of Copilot services, and the deployment of generative AI by numerous enterprises globally all hinge on NVIDIA's computational prowess.

Conversely, skeptics argue that the impact of proprietary chips has been overhyped. NVIDIA's true strength lies in its comprehensive ecosystem built around CUDA, and any competitor must confront the long-term challenge of software accumulation.

Similar endeavors in self-developed chips have already commenced in China. However, manufacturing the chip is merely the first hurdle. The software ecosystem, model adaptation, compilation optimization, cluster scheduling, and the overall operational efficiency of the computing system are equally critical in determining a chip's practical viability.

In recent years, the most significant transformation in the AI industry has been the rapid enhancement of model capabilities. With products like ChatGPT and Copilot entering the commercialization phase, the industry's focus has shifted.

Training large models demands substantial computational resources, but the long-term costs stem from inference. Training is akin to constructing a factory, whereas inference is the daily operation of that factory. As millions of users engage with AI assistants daily, computing costs continue to escalate.

Microsoft, with its Azure cloud services and as a key partner of OpenAI, reported that Azure's annual revenue has surpassed $100 billion. In the fourth quarter of fiscal year 2026, Microsoft Cloud revenue reached $59.3 billion, a 27% year-over-year increase; Azure and other cloud services revenue grew by 43%.

On the application front, the number of commercial users of Microsoft 365 Copilot continues to grow, with over 30 million paying users. Products like Copilot, GitHub Copilot, and Microsoft Foundry are driving up the demand for AI inference computing power.

In essence, Microsoft is developing the Maia 300 because Azure is facing mounting pressure from AI computing demands.

Following the surge of generative AI in 2023, high-end AI GPUs like NVIDIA's H100 became highly sought after by tech companies worldwide. The price of a single H100 card once soared to $25,000-$30,000, and during peak demand, delivery times were extended.

For ordinary enterprises, a single GPU is prohibitively expensive. For cloud computing giants like Microsoft and Google, the real challenge lies in scale.

Google utilizes TPUs to power its search, advertising, and AI services, while Amazon employs Trainium and Inferentia to support AWS.

Microsoft's Maia series, initiated with the Maia 100 announced in November 2023, has now reached its third iteration: the Maia 200, released in January of this year, is already in production and handling actual workloads for Microsoft services like Copilot. The Maia 300 is slated for large-scale deployment in 2027. The transition from Maia 100 to Maia 300 in less than three years marks a significant acceleration in iteration pace.

While the paths of these three companies may appear parallel, the nature of their investments has evolved. Proprietary chips are transitioning from an optional feature to a standard offering for cloud providers.

To paraphrase a comment from Satya Nadella's post: “Now everyone is building their own chips. The rule of the gold rush is: first sell shovels, then sell the factories that make shovels.” This metaphor holds some truth, but a more precise way to put it is that cloud providers are now selling the capacity of blacksmith shops and are picking up the hammers themselves.

The design focus of the Maia 300 centers on addressing three increasingly critical issues in AI computing.

Firstly, it aims to reduce data movement to enhance chip utilization. When large AI models operate, they constantly need to exchange data between computing units and high-speed memory. Insufficient data transmission efficiency leads to idle computing cores and decreased utilization.

Microsoft's software-defined data flow architecture for the Maia 300 is designed to tackle this issue. Simply put, it enables the chip to plan in advance where data should reside and when it should be accessed, minimizing the time computing units wait for data.

This is why Microsoft emphasizes “effective computing power per watt” rather than just peak computing power. Microsoft states that compared to leading GPU systems, the Maia 300 can deliver 40% more AI token processing capability per megawatt.

Secondly, high-bandwidth memory enables larger models to handle longer contexts. Future AI agents will need to simultaneously read enterprise databases, large volumes of documents, code repositories, and historical task records, requiring stronger data reading capabilities from the chip.

The Maia 300 single accelerator boasts a peak memory bandwidth of 18TB/s and a thermal design power of 950W. In a rack-level system, 96 Maia 300 accelerators can provide 27.6TB of HBM capacity and 1.7PB/s of aggregated memory bandwidth.

Thirdly, the Chiplet architecture reduces the manufacturing complexity of high-end chips. Traditionally, large chips integrated all functions into a single die; the Maia 300 separates computing, memory control, and I/O modules, enhancing design flexibility and reducing manufacturing costs. The computing chiplet utilizes TSMC's N2P advanced process, allowing for greater manufacturing flexibility by concentrating the most advanced processes in critical computing parts.

In addition to these three key points, the Maia 300 also reduces computing and memory demands during model operation through low-precision computing formats like MXFP4; improves multi-chip collaboration efficiency through high-speed interconnects and AI-optimized network protocols; and enhances overall data center throughput with a rack-level system composed of 96 accelerators.

These designs collectively aim to shift AI computing from a single-chip performance race to a system-wide efficiency competition.

This competition is even more pronounced in the Chinese market. In recent years, the domestic AI chip industry has diversified into multiple technological routes.

Huawei's Ascend series is constructing a general-purpose AI computing platform, having released the Ascend 910 and Ascend 310 AI processors in 2019. It then established an ecosystem around the MindSpore framework, CANN computing architecture, and Atlas hardware platform, enabling AI chips to penetrate cloud computing, industrial, energy, and other sectors.

Cambricon represents the route of professional AI chip companies. Its MLU series AI chips cover cloud training and inference scenarios. Compared to internet companies with extensive business scenarios, Cambricon focuses on addressing chip performance and software ecosystem construction.

Baidu and Alibaba, on the other hand, drive adoption through their own businesses. Baidu applies its Kunlun chip to AI businesses like search and intelligent cloud; Alibaba launched the Hanguang 800 inference chip, optimizing AI computing in conjunction with Alibaba Cloud services.

These companies aspire to establish a domestic computing power route distinct from NVIDIA's CUDA system. However, as large model applications deepen, a new question arises: once chips are available, how can they be effectively utilized?

For an AI chip to enter commercial applications, it must navigate multiple stages. First is the underlying hardware, including chip architecture, computing power, and memory bandwidth. Second is the software ecosystem, encompassing compilers, development frameworks, operator optimization, and model adaptation. Finally is practical application, where enterprises must migrate existing large models and business systems to the new computing platform.

A deficiency in any of these areas can compromise the chip's practical value.

Therefore, AI Infra (artificial intelligence infrastructure) has emerged as a new focal point in recent years. AI Infra addresses the efficient connection between chips, models, data, and applications. Chips provide computing power, software determines efficiency, and infrastructure determines whether this computing power can be truly harnessed.

Founded in 2023, InfiniteChip focuses on the AI Infra field. On October 12, InfiniteChip announced the completion of a $2.3 billion Series C funding round, bringing its total financing to $4.3 billion, setting a record for AI Infra financing in China.

The issues it addresses are highly aligned with global AI industry trends: enabling different chips to run different models; improving computing power utilization; and reducing the cost of AI deployment for enterprises.

Microsoft's approach to similar efficiency issues lies in its simultaneous ownership of models, chips, and workloads. Chip design can be directly optimized around its own workloads in reverse. Satya Nadella refers to this as a closed loop between models and chips.

Azure provides cloud computing services for enterprises while hosting OpenAI models, Microsoft's self-developed models, and Copilot series products. For example, training large models requires significant general-purpose computing resources, while products like Copilot generate a large number of inference requests daily, making them more sensitive to latency, cost, and energy consumption.

The Maia 300 is primarily aimed at these long-running AI inference tasks. Proprietary chips can be redesigned around their own businesses. Microsoft can independently decide which tasks to assign to GPUs, which to assign to Maia, and how to allocate computing resources.

However, Microsoft cannot completely abandon NVIDIA yet. Currently, Azure still extensively provides NVIDIA GPU services, and the CUDA ecosystem remains a cornerstone of AI development.

A more realistic scenario is that future data centers will simultaneously host multiple computing platforms, with NVIDIA GPUs handling general-purpose AI computing, Maia handling Microsoft-optimized AI workloads, and different chips being scheduled based on task requirements.

At least from Microsoft's own deployment plans, optimists' judgments still lack concrete evidence. The Maia 300 targets Microsoft's self-defined inference workloads, and Microsoft has explicitly stated that self-developed chips will not replace third-party supplies.

The real transformation lies in the nature of the procurement relationship: one of NVIDIA's largest customers is evolving into a significant competitor.