"Those who fail to plan for the whole cannot plan for a part."

At the WAIC Automotive AI Session on July 19, BYD, XPENG, and Li Auto unveiled their self-developed in-vehicle omni-domain large models. While the general-purpose large model sector continues to heat up, leading automakers are also focusing on dedicated in-vehicle AI, with cumulative R&D investments surpassing tens of billions. The collective entry of domestic automakers into self-developed large models signifies that in-vehicle AI has become one of the largest terminal scenarios for AI deployment.

However, a key question arises: Are automakers investing billions in self-developed in-vehicle omni-domain large models merely to optimize in-car voice interactions?

Omni-Domain Integration: In-Vehicle AI Goes Far Beyond Cockpit Upgrades

If you think of in-vehicle large models as just smarter voice assistants, you're underestimating their potential. Many vehicles on the market, equipped with basic large models, can engage in continuous dialogue and answer questions—clearly more than just voice assistants.

The true uniqueness of the new generation of in-vehicle omni-domain AI lies in its ability to integrate the cockpit, intelligent driving, and vehicle-road coordination seamlessly. Beyond independent entertainment functions, in-vehicle AI can aggregate road conditions captured by perception cameras, passenger states, and vehicle data into a single large model. While driving, the model can adjust driving strategies based on road conditions, regulate air conditioning temperatures according to passenger states, and anticipate risks through vehicle-road coordination. Previously, the cockpit handled multimedia, while intelligent driving focused solely on driving. Now, these two systems can collaborate to deliver a better user experience.

In the past, many automakers opted for external general-purpose large models to save costs and effort. However, these models were not designed for vehicles, suffering from slow response times, poor scenario adaptation, inadequate on-device computing power, and data security concerns. Due to these shortcomings, leading automakers have shifted toward self-developing lightweight, dedicated in-vehicle models to avoid a one-size-fits-all approach.

From this perspective, smart vehicles are indeed evolving into mobile intelligent spaces capable of perception and decision-making. However, investing billions to achieve large-scale deployment of in-vehicle large models still requires overcoming significant challenges.

What Old Problems Can Omni-Domain In-Vehicle Large Models Solve?

The smart electric vehicle industry faces several long-standing challenges.

First, the lack of coordination between the cockpit and intelligent driving systems is a major issue. Cockpit voice controls handle multimedia and climate, while autonomous driving systems operate independently, with minimal interaction between the two. Coordinating their functions requires separate commands.

Limited interaction capabilities also pose a key challenge. Traditional in-vehicle voice systems can only understand pre-set commands and struggle with complex scenarios.

While these issues are significant, the real problem lies in the inflexibility of on-device computing power. General-purpose large models are too bulky for vehicles, resulting in high costs and latency. Lightweight models, while more efficient, lack the necessary capabilities, making it difficult to find a suitable balance.

To address these challenges, automakers must develop self-developed omni-domain large models. An omni-domain architecture can integrate all in-vehicle data links, dynamically perceive internal and external environments, and adapt lightweight models to in-vehicle chips. Theoretically, this allows for integrated scheduling of the cockpit and intelligent driving systems.

These pain points are not new. The industry has previously attempted middleware and multi-system integration as solutions, but with limited success. The true uniqueness of omni-domain large models lies in their ability to handle everything through a unified model base, rather than relying on disjointed systems.

The success of this approach hinges on two key metrics: reducing on-device deployment costs and ensuring reliable model decision-making in extreme scenarios. Both issues can only be verified through large-scale real-world testing.

Visible R&D Investment vs. Invisible Deployment Costs

Many assume that with billions invested, automakers' products will undoubtedly outperform competitors. However, let's break down the costs. Developing a mature self-developed in-vehicle large model requires maintaining algorithm teams, purchasing computing power, and conducting long-term road testing and iterations. Billions are just the starting point, with ongoing annual investments needed for updates.

Beyond these visible costs, several hidden expenses exist.

First is hardware compatibility. Omni-domain large models demand higher computing power from in-vehicle chips, rendering older hardware obsolete. New models can only be installed in new vehicles, leaving existing owners without access to the benefits. Automakers must then devise ways to iteratively upgrade hardware solutions.

Second is the imperative of driving safety. While minor cockpit entertainment errors are tolerable, autonomous driving directly impacts passenger safety. Any reasoning errors or misjudgments by the large model in sudden road scenarios could have severe consequences. Thus, automakers must establish stringent safety systems to eliminate all potential errors in autonomous driving, as the stakes are irreversible.

Finally, there's the issue of stratified user experiences. Many casual drivers prioritize basic transportation needs, questioning whether complex features are truly essential. Over-integration of intelligentization (intelligent) functions could raise operational barriers, backfiring on usability.

Considering these hidden costs, it remains uncertain whether billion-dollar in-vehicle large models will align with consumer demands.

Automakers Entering the AI Arena: Rewriting Industry Logic

(Image sourced from the internet)

The release of in-vehicle large models may signal a transformation in the smart automotive industry landscape.

Traditionally, AI large models were the domain of tech companies, with automakers primarily serving as hardware manufacturers and procurers. However, as leading automakers like BYD, XPENG, and Li Auto invest heavily in self-developed models, they have become core players in the AI sector. In the foreseeable future, automakers will leverage vehicle terminals and vast real-world driving data to develop proprietary large models, creating unique advantages. They will no longer be mere AI users but technology developers.

Yet, competition in this sector is fierce. Autonomous driving tech companies like Horizon Robotics and Wumo Zhixing (Haomo.AI) are aggressively promoting their in-vehicle solutions, while general-purpose large model vendors are releasing lightweight vehicle versions. Although automakers hold an advantage in terminal scenarios, they still have significant ground to cover in accumulating large model algorithm expertise and technology.

Smart Cockpits as Mobile Intelligent Spaces: Full Adoption Still Awaits

The technological direction is clear—integrating cockpit, intelligent driving, and vehicle-road coordination to transform vehicles from transportation tools into mobile intelligent spaces is an industry-wide consensus. However, several hurdles must be overcome for large-scale adoption of in-vehicle AI. Automakers must balance computing power demands with manufacturing costs, establish robust safety systems to prevent model misjudgments, ensure compliance in data collection and usage, and identify genuine user needs rather than indiscriminately adding features.

The recent wave of unveilings at WAIC marks the official entry of the in-vehicle AI sector into a boom phase. With automakers investing billions, the race has begun. Key questions remain: When will in-vehicle large models enter mass production? How much will equipped models cost? Can long-term real-world testing ensure stability and safety? Answering these questions will determine whether in-vehicle AI is merely a marketing gimmick or a true game-changer for smart vehicle competitiveness.

These questions require time to resolve. For now, we wait and see.