What Obstacles Remain for Smartphones to Evolve from 'Being Operated' to 'Accomplishing Tasks'?

Over the past month, the most dynamic sector within smart hardware has undoubtedly been AI agent smartphones.

On July 15th, cybersecurity authorities issued, for the first time, an announcement concerning the filing information of seven services that provide on-device generative AI capabilities for smartphones. Subsequently, during the World Artificial Intelligence Conference (WAIC), the AI smartphone booths of Jieyue Xingchen and ZTE Nubia were swarmed with visitors eager to glimpse the future of smartphones.

Image Source: Qujie Business

At the heart of AI smartphones lies a pivotal technology—on-device AI. This technology enables large models to operate directly on the smartphone, eliminating the need to upload data to cloud servers for processing. Chat logs, photos, and documents can be comprehended and utilized by AI without ever leaving the device, thereby reducing latency and enhancing privacy.

Qujie Business highlights that, despite Jieyue and Nubia opting for distinct technological paths for their AI smartphones, on-device AI remains a focal point for both brands and all smartphone manufacturers moving forward.

01. AI Smartphones Can Accomplish Tasks, But Not Extensively Yet

Visually, the two AI agent smartphones from Jieyue Xingchen and Nubia are nearly indistinguishable from ordinary smartphones.

Jieyue's 'World's First Large Model-Native Agent Smartphone,' the STEPX Neo, sports an orange and black color scheme with dual rear cameras. Nubia's 'World's First Mass-Produced AI Agent Smartphone,' the NaviX Ultra, also known as the Doubao second-generation smartphone, is available in four colors: blue, pink, silver, and black, featuring a single-punch straight screen.

Honor unveiled the 'World's First Robot Smartphone,' the Robot Phone, which boasts a built-in movable mechanical gimbal capable of gesture wake-up, automatic subject tracking, and syncing with music rhythms. The intelligent agent solution, co-created by Alibaba and Honor based on the 'Qianwen Large Model,' was implemented on this device.

Interestingly, all three companies claim their products are the 'world's first' in their promotional materials.

Jieyue Xingchen also asserts to have developed the world's first agent-native operating system (Step AOS), which is not a deeply customized version of Android. However, its User Interface (UI) appears similar to Android, featuring a grid of app icons with orange and yellow rounded rectangles, though some third-party app icons differ slightly.

The most anticipated aspect of AI agent smartphones is whether they can transcend the graphical click-based interaction method. Currently, neither the Jieyue nor Nubia models offer a novel experience. According to on-site staff, the selling points of both smartphones mirror those of the previous Doubao first-generation smartphone, which caused a market stir—sending text or voice commands through a system assistant to schedule different apps to complete tasks.

Jieyue staff stated that the STEPX Neo can currently handle AI assistant-based ticket booking, food ordering, ride-hailing, file modification, and message replies on Feishu. Except for steps involving personal information and payment, which require user authorization, all other steps are autonomously completed by the AI assistant.

However, the back of the Jieyue smartphone features an interactive secondary screen (utilizing dot-matrix LEDs), hinting at future combinations of large and small screens that may support more diverse interaction methods, such as voice and vision.

Staff from both companies indicated that, in addition to the already partnered apps, all other apps on the market are in the process of being integrated. The launch dates for these smartphones remain undetermined, with many details still being refined.

Image Source: Qujie Business

This also implies that the future user experience of AI smartphones will hinge on the openness of major internet applications.

The Doubao first-generation smartphone adopted the 'GUI Agent' technical route (simulating user clicks), enabling it to call and execute apps without requiring open permissions. However, this approach led to it being "blocked" by apps like WeChat and Taobao due to security concerns. Perhaps due to these privacy and security controversies, neither Jieyue's STEPX Neo nor Nubia's model chose the 'GUI Agent' route this time. According to staff, Jieyue's STEPX Neo adopts a combination of GUI Agent and A2A, while Nubia's NaviX Ultra follows a pure A2A route.

A2A's hallmark is that it does not simulate user clicks but instead calls app capabilities through standard interfaces, with the apps executing the tasks themselves. This approach, also adopted by hardware giants like Huawei, OPPO, and Xiaomi, is more secure and stable but highly dependent on app permission openness.

From the current app partnerships, tool-based apps like Meituan and Ctrip are relatively proactive in integrating with smartphone AI, as their primary revenue comes from order commissions rather than advertising. However, high-frequency social apps like WeChat, Douyin, and Xiaohongshu, which rely on advertising as their core revenue, are reluctant to cede usage permissions, as it would be tantamount to surrendering their lifeblood (WeChat is even developing its own intelligent agent assistant).

Whether partnerships can be forged with these high-frequency apps will directly determine whether AI agent smartphones become 'useful tools' or 'expensive toys.' This remains the most critical variable to monitor.

02. Overcoming the 'Hardware Barrier'

Qujie Business notes that at these two smartphone booths, in addition to inquiries about 'how to operate,' visitors most frequently asked about task execution efficiency and data security.

On-device AI is perceived as the solution to these issues. The large models commonly used on web and app platforms, such as Doubao and DeepSeek, follow a cloud AI route. The logic of cloud AI is: user issues a command → data is uploaded to the cloud server → large model processes it → results are returned to the smartphone. In this process, users' chat logs, photos, location information, and consumption habits all pass through third-party servers, increasing latency and slowing down the agent's execution speed while also raising concerns about cloud service providers misusing, leaking, or retaining user data long-term.

The logic of on-device AI is entirely different: the large model is deployed directly on the smartphone, and user data, from generation to processing, never leaves the device, directly reducing latency and privacy risks.

However, due to hardware limitations, no smartphone can currently run a large model entirely on-device. All manufacturers currently opt for 'edge-cloud collaboration,' processing simple dialogues locally while still relying on the cloud for complex, long-chain tasks.

Recently, cybersecurity authorities disclosed the filing information for seven 'on-device generative AI services' for smartphones, belonging to Apple, Huawei, OPPO, vivo, Xiaomi, Samsung, and Nubia (ZTE). This marks the first time regulators have disclosed 'on-device AI' as a separate category, signaling the smartphone industry's sprint towards 'on-device AI.'

Image Source: Cybersecurity Administration

Smartphone manufacturers commenced research on on-device models in 2023, and over the past three years, the companies mentioned in the filing have sequentially released their latest achievements.

In June this year, Apple unveiled two on-device models at WWDC, including the AFM 3 Core Advanced, which boasts a total of 20B parameters but activates only about 4B per inference, balancing performance and efficiency, primarily for basic text generation and instruction following. Similarly, in the first half of the year, Xiaomi released the MiMo-V2 series, with a staggering 1T total parameters and 42B activated parameters, suitable for more complex reasoning, copywriting generation, and in-depth dialogues.

Different manufacturers deploy models of varying sizes on-device, with different expectations for the problems they aim to solve. Li Dahai, CEO of Minbai Intelligence, a company focusing on on-device large models, stated that on-device models cannot be evaluated solely by parameter size or benchmark scores; capability, speed, power consumption, and memory usage must all be considered.

Li also mentioned that smartphone manufacturers typically assess on-device models based on their adaptability to chips and reasoning efficiency. 'Users won't accept an AI feature that seems capable but noticeably drains battery, overheats, or responds unstably. So, under similar effectiveness, whoever can deliver the experience with lower power consumption and latency has the advantage.'

Hardware poses a significant barrier for on-device AI. On-device AI requires substantial parallel computing, but the CPUs and GPUs in smartphone SoC chips struggle to meet the computational demands of on-device AI, often necessitating dedicated NPUs for AI reasoning.

An investor focusing on consumer electronics noted that the current mainstream solution for on-device AI chips is to integrate NPUs within the SoC, a route taken by Apple and Qualcomm. There are already twenty to thirty companies in China developing on-device AI chips, each with a different technical approach. Future smartphone SoC designs may undergo significant disruption, shifting from a CPU-centric to an NPU-centric architecture, with CPUs, GPUs, and various acceleration units working in tandem.

Image Source: Canned Food Gallery

Additionally, on-device AI places higher demands on smartphone memory bandwidth and storage. For instance, only iPhones equipped with A17 chips or later, such as the iPhone 15 series and above, can run large models on-device. Given the rising memory prices, the overall cost of smartphones may continue to increase.

Overall, AI smartphones are still in the transitional phase from 'chatting' to 'accomplishing tasks.' Before they can be frequently used by consumers, numerous issues must be addressed, including app permission barriers, model capabilities, and chip computing power, all of which are critical bottlenecks.

However, in the long run, the smartphone industry, which has been stagnant for years, is poised for a wave of renewal. The winners this time may not be the same as last time. As smartphones begin to operate autonomously, not only will interaction methods be rewritten but also the power dynamics of the AI era.