What is the real market demand for AI infrastructure? How should high-quality token facilities be built?

“The amount of high-quality tokens you have determines how many you can sell,” a senior Baidu executive told Shuzhi Qianxian. Global internet giants have invested over a trillion US dollars in crazy (Note: retained as is for context, but likely means 'aggressively') expanding infrastructure, causing stock prices of chip, storage, and optical communication companies to surge several times over in a few months.

However, on the other side of this frenetic expansion lies anxiety about overcapacity. In early July, news that Meta was preparing for a cloud computing business and planning to sell potentially idle computing power caused thousands of billions of dollars in market value to evaporate from related sectors. Yet, just days later, Meta announced an investment of approximately 9.1 billion US dollars to build its first AI data center in Canada, one of the largest outside the United States. A senior industry insider told Shuzhi Qianxian, “They have to build it; whoever doesn’t build it now will fall behind,” but no one can accurately predict the real demand one or two years from now, so Meta has adopted a hedging strategy.

These two somewhat contradictory situations point to the same question: What is the real market demand for AI infrastructure? How should high-quality token facilities be built? At the 2026 Open Compute Conference held in Beijing on July 9th, upstream and downstream players in the industry chain released a series of new signals around this issue.

01 Multi-Agent Systems and Multi-Model Integration Are Changing the Logic of Infrastructure Development

If the question of “whether to build” is a strategic and capital-level game, then the question of “how to build” must first be answered by examining changes in demand. In the first half of this year, three new trends emerged in the AI application market:

Trend 1: AI coding has successfully commercialized, with general-purpose assistants expanding the user base from tens of millions to hundreds of millions, triggering an explosion in token production demand. Industry estimates suggest that AI coding currently accounts for 70%–90% or more of total model token consumption and is unlikely to change significantly in the next two to three years. This makes it the first large model technology to achieve commercialization.

Domestic giants are doubling down on AI coding, with “competition accelerating in the second half of the year.” Building on this foundation, major companies have launched general-purpose assistants this year, similar to Anthropic’s Coworker. These assistants, still powered by coding at their core, are expanding their reach from 30 million domestic code engineers to hundreds of millions of computer users.

AI coding typically calls on the largest flagship models, which are sensitive to latency. As the user base grows from tens of millions to hundreds of millions, the challenge is how to reduce the inference costs of the most powerful models while stably serving hundreds of millions of users. Essentially, AI infrastructure needs to shift from “providing GPU computing power” to “continuously producing high-quality tokens.”

Trend 2: Multi-agent systems are entering a period of rapid growth, offering a path to enhance model intelligence beyond traditional scaling laws. From Doubao and Kimi to Tencent’s recently launched general-purpose assistant WorkBuddy, these systems are powered by hundreds of expert agents spanning multiple domains.

The difference is palpable when using Kimi: for planning a team-building event, a single-agent version might overlook issues like venue closures or budget overruns, while a multi-agent version deploys dozens of agents to cross-check venues, transportation, and budgets, producing a far more reliable plan.

Yang Zhilin, founder and CEO of Kimi, stated that once a large model’s intelligence reaches a certain threshold, multi-agent systems become the key to scaling performance. By engineering agent clusters, task complexity and output quality can be enhanced without altering the underlying model’s capabilities. K2.6 already supports over 300 agents working collaboratively. Anthropic revealed a similar architecture last year: using Opus 4 as the “main agent” and deriving multiple Sonnet 4 “sub-agents” for parallel exploration, improving user search performance by 90.2% compared to a single Opus 4, albeit with token consumption 15 times higher than ordinary conversations—essentially “trading computing power for quality.”

Not only do single models and applications use multi-agent systems to enhance token output quality, but individuals and enterprises are also adopting more agents. IDC data shows that the overall agent market will grow at a compound annual rate of over 110% from 2025 to 2028. Gartner predicts that 40% of enterprise applications will integrate agents by 2026. The explosive growth in agent numbers means that call volume, concurrent pressure, and resource consumption will far exceed the scale of single-model Q&A.

Trend 3: Multi-model integration has emerged as another path to improve token quality. While single-model capabilities were once prioritized, Zhao Shuai, deputy general manager of Inspur Information, explained that new practices show AI systems are moving from single models to multi-model collaboration, delivering higher intelligence than any single model.

Inspur Information conducted internal validations of multi-model integration (AI Fusion): a single task is parallelized across multiple candidate models, with a review model refining consensus, identifying differences, filling gaps, and fusing results. Test data showed the integrated model topped a deep research benchmark with a score of 53.9% and outperformed single models in AIME2026 math reasoning and GPQA Diamond high-difficulty Q&A.

Zhao Shuai further clarified that “multi-model integration” does not mean running every task through a stack of models. Instead, tasks are routed by difficulty: simple Q&A goes to fast, efficient models, while code assistance and complex logic are handled by integrated models. For example, a market insight report might use a small model with billions of parameters for OCR recognition, a hundred-billion-parameter model for internal RAG retrieval, and only invoke a trillion-parameter model for the final synthesis of dozens of information sources.

“If the previous competition was about single-model capabilities, the next phase of AI adoption across industries will require multi-model integration and multi-agent ‘swarm intelligence’ to produce higher-quality tokens,” Zhao Shuai summarized.

02 AI Infrastructure Is Undergoing Synchronized Reconstruction

The “swarm intelligence” enabled by multi-model integration and multi-agent systems is also reshaping the structure of AI infrastructure.

Traditionally, GPUs and CPUs had distinct roles: GPUs handled training and inference, while CPUs managed cloud, data, and AI services. However, Zhao Shuai observed that the industry must now integrate the two because “agents must run on CPUs.”

“Agents are designed to solve tasks, which requires not just inference but also task decomposition, orchestration, tool invocation, execution, and memory management—all running on CPUs,” explained Chen Jian, chief cloud server architect at Alibaba Cloud. “The importance of CPUs is returning, and we’re entering an era of CPU-GPU collaboration.”

Evidence of this shift is seen in global giants ramping up CPU general-purpose computing power: Microsoft developed its own general-purpose server CPU, Google expanded its partnership with Intel, and server CPUs from Intel and AMD faced supply shortages, price hikes, and extended lead times.

A new division of labor is emerging: GPUs handle “thinking” by continuously generating tokens, while CPUs handle “acting” by orchestrating agent scheduling, tool invocation, and sandboxed execution for hundreds or thousands of agents. However, this raises challenges: how to match, connect, and share storage between CPU-based agent clusters and GPU-based inference clusters? This may require bringing agent clusters closer to GPU clusters, reconstructing the deployment logic of AI infrastructure—even involving rethinking China’s “East Data, West Computing” strategy, as simply placing GPUs in the west and business operations in the east near users may no longer suffice.

New challenges also arise in agent orchestration and model routing. Liu Song, vice president of TiDB, noted that “hundreds of millions of agents will be running” in the next two to three years. If memory, context sharing, and orchestration systems cannot keep pace, multi-agent systems may create new bottlenecks instead of reducing workload. “After AI coding, engineers are busier because everyone becomes a context transporter,” he said. This is why domestic internet companies have achieved near-10x efficiency gains in new applications but struggle to exceed 20% efficiency improvements in private deployments.

This collaboration is also reflected in physical specifications. Overseas, megawatt-scale GPU cabinets have become “de facto product standards with bulk supply,” while domestically, GPU cabinets reaching 300 kW by the end of this year are inevitable. If CPUs remain at old specifications of 5 kW or 10 kW per cabinet, they cannot match the infrastructure of intelligent computing centers.

This has led to new CPU server cabinet specifications. For example, Inspur Information launched a CPU-native liquid-cooled solution supporting 384 general-purpose processors per cabinet and up to MW-level power consumption, aligning CPU-side computing density with GPU-side power supply and cooling standards. Only with unified infrastructure can facilities support over 40,000 agents running concurrently.

On the GPU side, “multi-model integration” (Fusion AI) has evolved from last year’s vision of “running multiple trillion-parameter models on a single machine” to a deployable solution, driving the popularity of super-node AI servers.

Chen Xuanhao, product director at Qinghong Electronics, noted that super-node card counts “will reach thousands or more this year and in the future.” Zhao Shuai introduced Inspur Information’s Yuanbrain SD200 super-node AI server, launched last year, which features 4TB unified memory and can simultaneously run four trillion-parameter models or dozens of smaller models, switching combinations as needed. The enterprise version released this year supports 16 cards, lowering the computing power threshold for enterprises to use trillion-parameter models. “AI coding for security vulnerability detection and code writing now delivers value from trillion-parameter models, with far better efficiency than hundred-billion-parameter models,” he said.

Beyond CPU-GPU collaboration, model companies like OpenAI and DeepSeek are developing ASIC chips. Inspur Information’s technical experts explained that the “integrated architecture” proposed three years ago is a universal technology that interconnects CPUs, GPUs, memory, and SSDs via a high-speed network—“chips are just computing units.” Future token production will become increasingly granular: Prefill stages may use ultra-scalable architectures, while Decode stages use super-node architectures, further subdivided by Attention and FFN layers to match different chips. This area is currently under joint innovation with model and chip vendors.

An example of the benefits of hardware-software co-innovation: on the Yuanbrain SD200, Kimi’s K2.6 trillion-parameter model achieved a single-token generation time of 4.77 ms, the first to break the 5 ms barrier.

“Whether ‘swarm intelligence’ can further reduce industrialization costs and deliver greater value to customers remains the next challenge,” Zhao Shuai said. Hardware infrastructure must continuously adjust interconnectivity, ratios, memory, and latency to ensure everything is combinable, connectable, and shareable. “This is what makes computer architecture so fascinating.”

These advancements rely on the support of integrated architecture technologies. All ongoing infrastructure transformations point to a larger trend: the rise of gigawatt-scale AI computing centers.

03 Gigawatt-Scale AI Computing Centers: The Next Battleground for Infrastructure

As token demand grows, agent numbers increase, and multi-model integration deepens, AI infrastructure is evolving toward higher density, power, and system efficiency: gigawatt-scale AI computing centers.

Gigawatt-scale AI computing centers were a hot topic at the Open Compute Conference, with related technical forums drawing massive crowds.

“Megawatt-scale cabinets enabled gigawatt-scale AI computing centers, or rather, the need for gigawatt-scale centers drove the development of megawatt-scale cabinets,” Zhao Shuai said. The driver is the expansion (Note: likely means 'explosive') growth of large model training scales. Training AI models now requires tens of thousands or even hundreds of thousands of cards. When each card consumes 2,000 watts, gigawatt-scale centers become necessary, as existing data centers cannot accommodate such power demands.

To address these comprehensive reconstruction challenges, OCTC established the GW-Scale Open AIDC Working Group last year and released the industry’s first “GW-Scale Open AIDC Framework Technical Report” at this year’s conference, systematically defining the full-stack path for gigawatt-scale open intelligent computing centers.

The first area impacted by this transformation is power supply infrastructure. Zhao Shuai told Shuzhi Qianxian that megawatt-scale cabinets require significant changes to power links and facilities, necessitating 800V high voltage directly into cabinets. Previously, 380V entered cabinets, supporting 40–50 kW per cabinet. At hundreds of kilowatts, copper cables became too thick for practical use, while high voltage resolves this with lower current and thinner cables. Power links are simplified, with 800V entering data centers and cabinets directly, enabling the construction of efficient gigawatt-scale data centers.

Xie Wei of Flex further noted, “We’re now seeing genuine 1 MW-scale cabinets.” When power exceeds 1 MW, a single cabinet can no longer house the power supply, leading to the emergence of Sidecar bypass power cabinets. However, high voltage introduces new challenges: while 48V was once considered safe for human contact, “800V is dangerous voltage,” requiring stricter insulation, safety, and anti-interference measures. “The 48V ecosystem took two to three decades to mature, while the 800V ecosystem is still in its infancy,” Xie Wei said, emphasizing the need for industry-wide collaboration.

Cooling systems are also being Refactoring (Note: means 'reconstructed'). Xie Wei observed that liquid cooling dominates beyond 250 kW, and Li Jinbao, product manager at AVIC Optoelectronics, believes liquid cooling will evolve into native system designs as XPU and cabinet power continue to rise. Zhao Shuai predicted that “90% or even 100% of cooling in future data centers will rely on liquid cooling,” with large air walls in machine rooms supporting row-based cabinet cooling—a key direction outlined in the gigawatt-scale open intelligent computing center white paper.

Interconnectivity is also evolving. Gao Xiaojun, server architect at ByteDance, expects XPU power to reach 2,000–3,000 watts in the next 2–3 years, making “optical interconnects in Scale-up architectures a necessity.” Chris Petersen, a board member of the UALink and CXL alliances, noted that collection communication pressures from MoE models and large-scale XPU clusters are becoming bottlenecks for computing power release. Copper and optical interconnects are forming a layered collaboration based on distance, power, and density. Chen Xuanhao of Qinghong Electronics believes copper interconnects will remain fundamental for intra-cabinet connections, while Peng Xiaowei of Luxshare Precision Industry sees integrated opto-electro-thermal designs as critical for breaking communication bottlenecks.

In summary, the signals from this Open Compute Conference are clear: New trends like AI coding, multi-agent collaboration, and multi-model integration are driving systemic reconstruction of AI infrastructure. CPUs and GPUs are shifting from division of labor to collaboration, super-nodes are moving from concept to scale deployment, and gigawatt-scale AI computing centers are accelerating as the new industry direction.

These three signals all aim to address the shortage of high-quality tokens. As the infrastructure supporting them is being rebuilt, industry participants agree that while the market is flourishing with diverse solutions, only through open collaboration can foundational technologies like high-speed interconnects, power supply, and cooling truly synergize to support the rapid deployment of agent-based infrastructure.