The 'State of the Processor Industry 2025' report from the Yole Group marks a significant milestone: GPU sales surpassed those of CPUs for the first time in 2024. This event signifies the semiconductor industry's formal entry into a new era dominated by accelerated computing. Amidst this macroeconomic backdrop, a pivotal question arises: As GPUs, NPUs, and ASICs—collectively termed 'accelerators'—gradually assume responsibility for large-scale parallel computing tasks, will traditional CPUs be relegated to the periphery? Or can they carve out an indispensable niche in the new computing paradigm?

The shift in focus within the GenAI computing landscape has not diminished the necessity for host processors but has fundamentally redefined performance benchmarks. Over the past three decades, CPUs have primarily propelled Moore's Law by enhancing general-purpose computing frequencies and speculative execution efficiency. However, when confronted with the high-throughput demands of training trillion-parameter models and real-time inference, this general-purpose design philosophy faces dual challenges of energy inefficiency and I/O bottlenecks.

The industry is reassessing the role of CPUs within AI clusters. Traditionally, they functioned as simple logic controllers; now, they are evolving into scheduling hubs for heterogeneous systems, offering not only substantial memory capacity but also directly managing specific inference tasks. This transformation is not only reshaping underlying technical architectures but also profoundly influencing market dynamics and capital flows, extending from data centers to edge devices.

01

CPU Dilemmas and the Path to 'Transformation'

In conventional CPU-centric computing architectures, data processing workflows are typically orchestrated by software stacks running on the CPU, necessitating multiple data transfers between network interfaces, CPU memory, and deep learning accelerators (DLAs). This software-based data management mechanism exposes significant efficiency flaws when handling AI workloads. The primary issues stem from parallel command conflicts and data path congestion, which directly constrain backend accelerator utilization, leaving expensive hardware resources idle and driving up overall system power consumption and costs.

A deeper technical contradiction lies in the design philosophy of processor microarchitectures. Modern CPUs extensively rely on 'speculative execution' technology, employing branch prediction to pre-execute instructions and maintain pipeline fullness. This mechanism excels in handling logically intricate general-purpose programs. However, AI and machine learning workloads predominantly consist of large-scale vector and matrix operations, with memory access patterns often exhibiting high irregularity. In such scenarios, speculative execution is prone to prediction failures, leading to frequent pipeline flushes. Discarded computational instructions not only fail to yield meaningful output but also incur additional energy waste and latency.

To overcome the limitations of general-purpose architectures under AI workloads, the processor industry is undergoing a foundational innovation: de-speculation at the microarchitecture level. The recently patent-certified 'Time-Based Deterministic Execution Model' by the U.S. Patent and Trademark Office represents a novel design approach. This model abandons complex speculative mechanisms and introduces vector coprocessors with time counters, adopting a static scheduling strategy. Under this architecture, instructions are dispatched to execution units only when data dependencies are fully resolved, and operands are ready at a predetermined moment.

Since execution order and timing are pre-planned and deterministic, chip design can eliminate complex register renaming and out-of-order execution control logic, achieving high scalability with lower transistor overhead and power consumption in matrix computation tasks. This deterministic execution model maintains compatibility with standard instruction sets like RISC-V while fundamentally adapting to AI computing's stringent demands for high throughput and low latency.

The second-tier innovation involves system-level architecture 'specialization and diversification.' To address I/O bottlenecks, the industry is exploring the separation of network ordering, quality of service (QoS) management, and data preprocessing functions from the host CPU's software stack, offloading them to dedicated hardware logic. This design concept, known as the 'Network-Attached Processing Unit' (NAPU), achieves hardware acceleration of data paths by integrating DSP cores, video engines, and AI-optimized network interfaces within the processor.

This not only liberates general-purpose CPU core computing resources for complex logical scheduling but also significantly reduces invalid data movement between components. Additionally, mainstream x86 processors are evolving through the integration of dedicated acceleration instruction sets like AMX, optimizing processing capabilities for low-precision data types such as bf16 and int8, thereby enhancing CPU efficiency in matrix operations without relying on external accelerators.

02

CPU Application Scenarios in the AI Era

The evolution of technical architectures directly mirrors structural changes in market demand. While training-end demand for GPUs continues to surge, the inference-end market is exhibiting sensitivity to cost and energy efficiency, creating broad market opportunities for new CPUs. According to Future Market Insights, U.S. data center CPU demand will sustain a 7.4% compound annual growth rate. This growth is not driven by demand for traditional general-purpose computing power but by the practical 'economic calculus' of AI application deployment.

In inference scenarios, not all tasks necessitate expensive GPU clusters. For numerous small-to-medium models with parameter counts ranging from 7 billion to 13 billion, or for real-time interactive requests from single users, modern server CPUs already provide sufficient throughput. Intel's data indicates that dual-socket servers can achieve token generation rates meeting real-time reading speeds when running Llama models of specific parameter scales.

More critically, according to AsiaInfo Technologies and Cast AI, a significant portion of idle CPU resources in public cloud environments operate at utilization rates below 20%. Leveraging these deployed general-purpose computing resources for AI inference offers substantial total cost of ownership (TCO) advantages compared to purchasing dedicated accelerators. Thus, in long-tail applications and non-high-concurrency scenarios, CPUs are emerging as viable AI inference workhorses, with this 'good enough' economic logic supporting sustained growth in the data center CPU market.

Beyond directly handling inference tasks, AI large models' demand for memory capacity is reshaping CPU market value. As model parameter counts surpass trillion levels, GPU memory capacity increasingly becomes a bottleneck for inference performance, with memory overflow leading to service interruptions. In this context, CPU main memory, shared with GPUs via high-speed interconnect technologies like CXL, effectively serves as an L4 cache for accelerators.

In NVIDIA's GH/GB series and Huawei's Ascend super-node solutions, high-performance CPUs paired with large-capacity DDR memory have become critical infrastructure for stable large model operation. This implies that market evaluation criteria for server CPUs are shifting, with memory channel count, bandwidth, and interconnect speed with accelerators becoming more crucial selection metrics than core frequency.

Expanding the perspective to edge computing and terminal devices, market demand for 'heterogeneous collaboration' has surpassed single-chip performance. In embodied intelligence and smart terminal domains, system designs feature a strict division of labor: CPUs handle low-latency logical control and real-time interaction, GPUs manage high-concurrency computing, and NPUs process continuously running background tasks.

Industry experts note that in scenarios like speech-to-text conversion, complex logical scheduling, and real-time motion control, CPU response speeds outperform GPUs, which require batch processing for efficiency. For instance, in robotics, x86 CPUs, leveraging their software ecosystem accumulation in industrial control, paired with embedded GPUs, remain the mainstream choice for master control solutions. This heterogeneous computing market trend necessitates stronger collaboration capabilities from CPUs, enabling efficient offloading of specific workloads to NPUs or GPUs while maintaining precise global task scheduling.

03

AI CPUs: Giants and Emerging Contenders

Driven by technological transformation and market demand, the competitive landscape of the processor industry is undergoing a reshaping. On one hand, startups specializing in AI-dedicated architectures are emerging; on the other, traditional giants are adjusting strategies and pursuing ecological integration.

Israeli chip company NeuReality exemplifies this specialization trend. The company recently completed a $35 million Series A funding round, bringing its total financing to $48 million, with a commercialization target directly aimed at the AI inference server market. NeuReality's NR1 chip represents a deconstruction and reorganization of traditional CPU architectures, defined as a 'Network-Attached Processing Unit' (NAPU). The chip integrates Arm Neoverse cores, but its core competitiveness lies in its heterogeneously integrated sixteen general-purpose DSP cores, sixteen audio DSP cores, and four video engines.

Through this hardware design, NeuReality attempts to address traditional CPU bottlenecks in processing AI data streams by solidifying network ordering, data sorting, and synchronization tasks in hardware. Its publicly released data indicates that compared to traditional CPU-centric architectures, the NR1 can enhance the total cost of ownership (TCO) for AI applications by tenfold. The emergence of such specialized chips signifies market acceptance of the idea that AI pipelines should no longer be dominated by general-purpose CPUs but managed by dedicated host processors.

Meanwhile, traditional chip giants are actively adapting to these changes, consolidating their ecological positions through capital operations and technical collaborations. In September 2025, NVIDIA announced a $5 billion investment in Intel and infrastructure cooperation, a commercial move with strong signaling value. Despite NVIDIA's dominance in accelerated computing, its substantial investment in the x86 ecosystem indicates that for the foreseeable future, high-performance x86 CPUs will remain strategically valuable as the universal foundation and ecological gateway for heterogeneous clusters. This is less a compromise than an acknowledgment of reality—no matter how powerful GPU clusters become, they still require capable CPUs for orchestration.

On another front, the Arm architecture is mounting a strong offensive in the server domain. Data indicates that Arm-based CPU market share in servers continues to climb, projected to account for 21.1% of global server shipments by 2025. This growth is driven not only by cloud vendors like AWS developing their own Graviton series chips but also by manufacturers like Fujitsu expanding in the European market. Fujitsu's strategic cooperation with European cloud service provider Scaleway aims to build energy-efficient AI inference environments using the Arm-based FUJITSU-MONAKA CPU platform, bypassing the red ocean competition for GPU computing power and seeking breakthroughs in green computing and low-TCO inference.

However, internet giants' actual deployment strategies also reveal market complexity. Despite cost advantages of self-developed Arm chips, x86 CPUs remain the preferred configuration for core AI training clusters to ensure absolute software ecosystem compatibility and stability. This indicates that the future processor market will not be a zero-sum game between single architectures but will enter a complex ecological phase where x86 and Arm coexist, general-purpose CPUs and dedicated AI CPUs complement each other, and CPUs and accelerators engage in deep collaboration.

In this landscape, CPU vendors' competitiveness will no longer hinge solely on core count or frequency but on whether their architectures are sufficiently open, whether they can efficiently integrate into heterogeneous computing pipelines, and whether they can provide the most cost-effective computing support for increasingly diverse AI workloads.