By Liang Tian

From Node Finance

In recent years, the AI industry has been dominated by a frenetic race driven by the Scaling Law.

Data centers are brimming with GPUs, and model parameters have soared past the trillion mark, with players aggressively investing in computing power, data, and algorithms.

A pressing issue has emerged: these colossal systems are confined to data centers, while in reality, various terminal devices are the closest touchpoints with consumers.

Regrettably, a physical gap separates computing power from practical applications.

Fortunately, a decentralized trend is gaining momentum.

AI-powered phones, AI PCs, smart cars, robots, and even AI glasses, equipped with compact on-device models, are experiencing a surge. Li Dahai, CEO of Facemind, identifies 2026 as the inaugural year for the commercialization of on-device AI. Gartner predicts that by 2026, AI PCs will account for half of the global PC market; Counterpoint also forecasts that smartphones with generative AI capabilities will capture 45% of the global market share in the same year.

New product categories are emerging, with the most advanced AI systems remaining in the cloud, while high-frequency, low-latency tasks closely tied to personal privacy are shifting to terminals.

Why are models starting to migrate from the cloud?

The distinction between cloud AI and on-device AI can be succinctly summarized as outsourcing versus integration.

Renowned models like DeepSeek, Kimi, and ChatGPT are predominantly large-scale, with parameters in the hundreds of billions or even trillions. Access to these models typically requires purchasing API keys, and even free chatbots are essentially model factories that encapsulate models as chatbots. The inference process is not completed locally; instead, devices send requests to the cloud, where GPU clusters in data centers perform the computations and return the results.

In contrast, on-device AI models are significantly smaller, typically with parameter counts in the billions. The inference process is completed offline directly on the terminal device, effectively endowing the device with a locally deployed brain.

While on-device models may not match the intelligence of large cloud models, their lightweight nature and local deployment offer distinct advantages in terms of cost, latency, and privacy.

Consider an AI Agent that operates continuously, helping you plan trips, organize meeting minutes, and even respond to emails. While convenient, a single task may involve numerous steps. If each step requires a round trip to the cloud, the accumulated latency will degrade the user experience, and the continuous consumption of tokens will result in exorbitant computing costs.

Privacy is even more critical.

When Agents truly integrate into PCs and phones, they no longer handle public data on the internet but deal with users' private photos, account permissions, and chat content. This information is highly sensitive.

Should data leave the device? This widely debated privacy concern looms over cloud model vendors like the Sword of Damocles.

You might argue that since large models are open-source, they can simply be deployed locally. However, even local deployment of large models demands immense computing power, making them accessible only to enterprise-level players. For consumer-end (C-end) users, a combination of local small models and cloud-based large models is the practical solution that balances capability, privacy, and cost.

This raises the question: If on-device AI is so convenient, why has its prominence only risen years after the rise of large models? Because deployment is no trivial task.

Why the rush into terminal devices now?

Integrating small models into compact smart devices is no small feat.

All models require chips for computing, storage, and communication. While several GPUs today can support the inference of large models with hundreds of billions or trillions of parameters, deploying chips in compact consumer devices (C-end) for small model inference remains a challenge.

If chips are expensive, they can support intelligent models but often come with prohibitive costs and high power consumption, making them unsuitable for space-constrained PCs and phones. If chips meet power consumption standards, they may fail to ensure intelligence.

Overall, on-device AI faces an insurmountable trilemma of performance, power consumption, and cost for deployment and commercialization.

Fortunately, advancements in underlying hardware and algorithms over the past few years have found a balance among these seemingly irreconcilable trade-offs.

First is the upgrade in computing power architecture.

For on-device AI, speed alone is insufficient; reducing the massive power consumption caused by data reading and writing is a crucial internal competition among chip vendors. The maturity of NPU technology is making it a standard feature in consumer electronics. For example, vendors like Qualcomm, Rockchip, and Huawei have begun optimizing for inference scenarios in terminal models, reducing data movement through shared memory and even adjusting chip architectures specifically for long-context and Mixture-of-Experts (MoE) models, providing comprehensive services to more model factories through hardware-software integration.

Second is the leap in model capability density.

Early small models were often associated with limited intelligence, but now, on-device models can pack a punch despite their small size. Vendors enhance small model capabilities through distillation, reduce model storage and computation overhead through quantization, and further compress long-context inference costs with techniques like sparse attention.

The Density Law proposed by Tsinghua University and the open-source community OpenBMB states that the capability density of large models grows exponentially over time, doubling approximately every 3.3 to 3.5 months. In other words, every hundred days, a model with half the parameters can achieve the current optimal performance.

Take Facemind's open-source MiniCPM5-2B as an example: with only 2 billion parameters, it leads the small model track in evaluations and even possesses tool invocation and code generation capabilities.

Finally is the bottom-up reconstruction of operating systems.

If on-device AI remains confined to a dialog box, it will always be just a novelty. For Agents to truly add value, they must have permissions to read system context, access cameras, and invoke third-party apps.

Step AOS by Step.fun and vivo's Blue Heart Large Model are both attempting to break down this barrier, transforming thousands of atomic capabilities of operating systems into AI-callable tools.

Chips provide the computing foundation, models push the capability ceiling, and operating systems grant execution permissions. The technological closed loop of on-device AI has finally come together.

Who will reap the rewards of on-device AI?

On-device AI is not a single product but is distributed across chips, models, infrastructure, and terminals. As AI capabilities shift from the cloud to terminals, China's on-device AI market has begun to diverge into different routes.

From Node Finance's perspective, controlling key layers grants different pricing and distribution powers. On the surface, vendors are all betting on on-device AI, but upon closer inspection, their strategies differ.

The first category is model suppliers.

The core logic of these companies is to make models small enough, powerful enough, and adaptable to as many hardware platforms as possible.

Facemind abandoned thousand-billion-parameter cloud-based models in 2023, fully pivoting to on-device AI and proposing the concept of "capability density." Its MiniCPM series continues to iterate toward smaller parameters and higher capabilities. The MiniCPM5-2B, open-sourced on September 7, has only 2 billion parameters, primarily targeting local deployment on phones and PCs while strengthening Agent capabilities.

Unlike phone or car manufacturers, Facemind does not control terminals; it aims to be an independent third-party model supplier, deploying the same on-device model capabilities across different devices like phones, cars, and robots.

Currently, MiniCPM has been integrated into Samsung Galaxy AI's on-device capabilities, and the car-end intelligent cockpit assistant cpmGO has gone into mass production with Chang'an Mazda's MAZDA EZ-60. In embodied intelligence, MiniCPM-Robot has entered scenarios like exhibition hall guidance and campus inspections.

The competitiveness of these companies lies not just in model benchmarks but in capability density, quantization compression, cross-chip adaptation, and model ecosystems. They must prove that an independent model company can also serve as an AI capability supplier for various hardware vendors.

iFLYTEK is also promoting small model on-device deployment but follows a different path.

On September 1, iFLYTEK open-sourced two on-device models, Spark X2.5-4B/1.7B, supporting contexts up to 1 million tokens and focusing on AI office scenarios, where it already has advantages.

Unlike Facemind's attempt to be a horizontal model supplier, iFLYTEK is more akin to using on-device models to strengthen its existing products in voice, office, and enterprise productivity.

The second category is chip players.

Vendors like Rockchip primarily provide SoCs, NPUs, and supporting toolchains for different terminals like phones and robots. Their core value lies in deploying models onto hardware with acceptable power consumption, cost, and performance.

Compared to this relatively generic chip supply model, Horizon Robotics extends further into software and scenarios. It not only provides Journey series chips but also offers advanced driving assistance systems, models, and software stacks, making it closer to a software-hardware integrated computing platform. Its competitiveness comes not just from chips but from the collaborative optimization of chips, models, and algorithms.

The third category is Agent entry point players.

For example, the currently popular Doubao phone.

In traditional smartphones, users open an app to complete tasks like hailing a ride, ordering food, or making payments. In the Agent era, users may only need a natural language instruction to let AI complete task invocation across different applications.

This is also the important context behind Doubao's collaboration with ZTE Nubia to advance AI phones. After the first-generation Doubao phone, the Nubia M153, the two sides shifted the solution from cross-app operations to on-device MCP, hoping to enable Agents to invoke phone capabilities more deeply. The second-generation Doubao phone, NaviX Ultra, attempts to complete tasks like hailing a ride and ordering food through voice Agents while adding visual perception capabilities.

Although the currently callable third-party services are still limited, and high-permission scenarios like payments are constrained by security, their strategic direction is clear: The phone is just a carrier, and installing models into phones is ByteDance's strategy to compete for the C-end entry point in the Agent era.

The fourth category is vertically integrated players, represented by Step.fun.

Unlike Facemind, which primarily provides models, or Doubao, which competes for entry points, Step.fun aims to further connect foundational models, Agent systems, and hardware terminals.

On July 13, Step.fun released the terminal brand STEPX and the Agentic Phone STEPX Neo. Meanwhile, the company collaborated with Qianli Technology and Geely in the automotive sector and promoted Step AOS into vehicles.

This means Step.fun's goal is not just to sell models to device vendors but to extend from foundational models all the way to Agent operating systems and terminal products.

It represents another route for on-device AI: through vertical integration, it seeks to control how models perceive the environment, invoke tools, and ultimately integrate with hardware like phones and cars.

The above routes all have certain technical barriers, leaving enterprises overwhelmed by questions like which chips or models to use for their business and how to integrate them with their ecosystems. Therefore, delivery-focused players have emerged in the market. Clients submit requirements, and these players combine the elements into a deliverable solution.

For example, in safety-critical scenarios like automobiles, Desay SV can decide which functions to place locally and which tasks to offload to the cloud based on vehicle price, chip computing power, and client needs, providing different specifications of models and computing solutions.

Compared to large models pursuing the upper limit of intelligence, on-device AI issues lean more toward engineering optimization, determining whether on-device products are just demos for showcasing at product launches or truly life-integrated products.

Over the past three years, the cloud has proven how intelligent AI can be, but being the smartest does not necessarily mean being the closest to real-life scenarios. Next, on-device AI must prove how to balance intelligence, cost, and power consumption to truly become part of devices.