Foreword:

The narrative of edge AI is built on an enticing premise: moving inference from the cloud back to local devices can save massive API bills every month.

This story held true from 2022 to 2024, became questionable starting in 2025, and fell apart by 2026.

The marginal cost of computing power is declining, while the marginal cost of the medium carrying that computing power is skyrocketing.

When a device requires 16GB or 32GB of memory just to fit the KV Cache of multimodal models, the growth in computing power numbers can never keep up with the surge in memory prices. Those impressive figures on manufacturers' spec sheets turn into subtraction problems when it comes to the bill of materials (BOM).

Author | Fang Wensan

Image Source | Internet

Inflation of Computing Power Metrics Degrades Judgment

Opening the presentation slides for mobile chips, the fifth-generation Snapdragon 8 Supreme, Dimensity 9500, and Apple A19 all boast NPU computing power exceeding 100 TOPS, claiming to run multimodal models with 4B to 13B parameters locally. The automotive sector is even more exaggerated: NVIDIA Thor initially claimed 2000 TOPS, while Horizon Robotics' Journey 6P is marked at 560 TOPS.

The bottleneck for edge inference has long shifted from multiply-accumulate operations to data movement. Thor's 2070 TFLOPS represent a theoretical peak under sparse FP4 conditions, yet its LPDDR5X memory bandwidth is only 273GB/s.

In real-world testing, the speed of large model text generation is almost solely determined by bandwidth, not computing power. It's like giving a sports car full throttle but equipping it with a straw-sized fuel line.

An even deeper misalignment lies in the conflict between two types of time.

Chip development, from project initiation (project initiation) to mass production, takes three to five years. The cost of 5nm-and-below design projects exceeds $500 million, with single tape-out fees around $30 million. Meanwhile, the iteration cycle for model architectures has shortened to months—just as Transformers gain a foothold, Mamba, Mixture of Experts, and vision-language-action models take turns Appearing (making appearances).

The edge AI chips shipping in 2026 sport impressive numbers on their spec sheets.

Qualcomm's Snapdragon X2 Elite NPU reaches 80 TOPS, with an extreme version hitting 85 TOPS. AMD's Ryzen AI 400 series delivers 60 TOPS, while Intel's Core Ultra Series 3 NPU offers 50 TOPS.

By 2026, this threshold has been easily surpassed by all mainstream platforms, rendering TOPS meaningless as a differentiator.

It also holds almost no predictive power for measuring large language model speed.

Autoregressive decoding is constrained by memory bandwidth, not computing power. In this equation, TOPS doesn't appear in any term.

In a 2026 comparison, Qualcomm's Snapdragon X Elite NPU delivers 453 TOPS with 136 GB/s bandwidth, while AMD's Ryzen AI 9 HX 370 offers 505 TOPS with 128 GB/s bandwidth. Despite a 52-point TOPS gap, their generation speeds fall into the same range.

AMD's Ryzen AI 300 takes about 70 seconds to generate one image using Stable Diffusion on its NPU, but only about 30 seconds on its integrated GPU. Under real workloads, the NPU is actually slower.

The dividing line has shifted to memory capacity and bandwidth. A notebook claiming 50 TOPS might only run large models locally at 9.4 tokens per second—not due to chip defects, but inevitable bandwidth constraints.

The cost of metric inflation is degraded judgment. When everyone chases TOPS numbers, purchasers lose the ability to discern differentiation, while manufacturers welcome this confusion because higher numbers justify higher premiums.

NVIDIA's Jetson Thor spec sheet provides an observational sample: its 2070 TFLOPS of FP4 sparse computing power is widely cited as a 7.5x improvement over the previous generation's Orin INT8 sparse computing power.

Similar vague practices are rampant in the industry. Qualcomm's Dragonwing IQ-2390 NPU offers just 1.1 TOPS, MediaTek's Genio 420 delivers 7.2 TOPS, and Jetson Orin NX reaches 100 TOPS—nearly two orders of magnitude apart yet often placed in the same comparison table.

AI is merely a feature, not the product itself. NPU-induced power consumption and costs far exceed expectations. A 5-watt module like Hailo-10H can handle inference for 7B-parameter models, while the $3,499 Jetson Thor enables real-time inference for 70B-parameter models. Both are correct choices within their respective operational ranges.

Chips Don't Make Money—Ecosystems Do

The cloud AI training chip market is dominated by NVIDIA, but the edge market tells a completely different story.

In 2025, even top-ranked Qualcomm held just a 15.6% share, with NVIDIA at 9.3%, Intel at 8.8%, and the remaining two-thirds of the market scattered among dozens of vendors.

This fragmentation stems from the incommensurability of scenarios: a always-on earbud requires microwatt-level keyword detection chips, an L3 autonomous vehicle needs 100–300 TOPS with ASIL-D functional safety certification, and a humanoid robot demands hundred-watt-class multimodal brains. No single chip can serve all three.

Automotive is one of the few scenarios achieving scale, making it the best sample for observing this war.

Horizon Robotics shipped over 4 million chip solutions in 2025, up 38.8% year-on-year, including 1.8 million mid-to-high-end chips supporting NOA (Navigate on Autopilot), nearly quintupling in one year and contributing over 80% of product and solution revenue. By the first half of 2026, Horizon captured 31.94% of China's self-branded intelligent driving chip market, surpassing NVIDIA's 29.38%, with urban NOA appearing in mass-market sedans priced at RMB 89,900.

Dramatically, NVIDIA's stance tells a different story. Its in-vehicle Thor chip delayed from 2024 to August 2025, with its advertised 2000 TOPS shrinking to 700 TOPS upon delivery. Automotive accounts for just 1% of NVIDIA's revenue, making this market's strategic priority for NVIDIA clearly capped.

Giants' edge businesses overlap with specialists' core territories—this attention mismatch defines survival logic in niche markets.

Shifting focus from product spec sheets to financial statements reveals a highly consistent strategy among giants: chips are merely hooks.

NVIDIA replicates its CUDA moat at the edge—Jetson hardware bundles the Isaac robotics stack, Omniverse simulation, and Metropolis platform, making developer migration difficult once entrenched.

Qualcomm acquired edge AI development platform Edge Impulse (with 250,000 developers) in March 2025, then purchased open-source hardware community Arduino (with 33 million users) in October, launching the $44 UNO Q development board.

Business models are quietly shifting. One-time hardware sales margins shrink, prompting vendors to bundle chips with software subscriptions: model version management, security patches, and device orchestration add 15–20% profit margins.

Startups caught in the middle face a delicate position. SiMa.ai has raised $355 million in total funding, with its 2026 funding round valuing it at approximately $1.4 billion. Hailo, Axelera AI, and EnCharge each bet on in-memory computing or specialized dataflow architectures, with EnCharge claiming its analog in-memory computing achieves 150 TOPS/W energy efficiency.

Capital continues to bet because no unified winner has emerged in the edge market—yet the window is narrowing. The speed at which giants acquire developer communities defines how quickly that window closes.

Opportunities Lurk Beyond Constraints

Fortune Business Insights projects the global edge AI semiconductor market will grow from $25.81 billion in 2025 to $107.86 billion by 2034. Grand View Research is even more optimistic, forecasting $291.8 billion by 2033 with a 34.7% CAGR.

The Chinese market is equally fierce. IDC predicts inference will account for over 70% of intelligent computing demand by 2027. In the first half of 2025 alone, China's edge computing market surpassed RMB 42 billion, up 28.6% year-on-year.

Constraints don't kill demand—they reshape it. Three shifts warrant attention:

① End-side agents are scaling: Institutions like ModelWall and Tencent Hunyuan compress models to 300MB–1.6GB, enabling AI assistants to run fully offline on phones and glasses. End-side devices now autonomously complete multitasking. Models shrink faster than chips grow, directly rewriting demand curves for computing power.

② Physical AI raises power envelope ceilings: The market for vision systems in robots and drones is expected to exceed $9 billion by 2033. Robots don't mind 100-watt power consumption—they care about determinism and development efficiency.

③ Storage crises are forcing architectural revolutions: When memory becomes more expensive and scarce than computing power, economic rationales emerge for in-memory computing and event-driven neuromorphic chips. Neuromorphic architectures are projected to grow at a 48.3% CAGR, several times faster than the overall market.

Epilogue:

Cloud-based large models reside in spacious data centers with electricity meters spinning freely. Edge-side chips, however, are allocated just a few watts of power while answering queries in cameras, steering wheels, and robot joints on the spot.

As AI moves from the cloud to everything, the toughest battle is fought on the smallest silicon.

Partial References: TrendForce: 'Mobile DRAM Price Surge Pressures Design Planning', UBS: 'Memory Chips Enter Pricing Upcycle', KB Securities: 'AI Boom Impacts Supply Chain', Counterpoint: 'Edge AI Statistics 2026: Market Size, Chips, Adoption & Sector Data', Three Artisans Report: '2026 Edge AI Scaling Paths: Four Thresholds Determine How Far LMIC Can Go'