Presently, numerous companies within the intelligent driving industry are channeling their efforts into scaling up cloud-based intelligent driving models, while simultaneously exploring methods to compress and implement these models on vehicles.

In September, XPENG revealed the latest iteration of its second-generation VLA, boosting the parameter count of the edge-side model to 3.5 times that of its predecessor. Concurrently, it reduced the computational burden on the vehicle side through techniques like learning-based token compression and distillation training. Simultaneously, the processing power of in-vehicle chips is on the rise.

These two seemingly paradoxical endeavors actually converge on a single question: As model capabilities expand, how much computational power is genuinely required on the vehicle side, and how can this power be harnessed effectively?

01 Why Must Processing Power Increase as Models Expand?

The growth in model size leads to a persistent surge in computational requirements.

Intelligent driving models have transitioned from relatively isolated perception tasks to end-to-end, multimodal, and VLA systems, necessitating the handling of increasingly intricate information.

Models must not only recognize vehicles, pedestrians, and roads but also comprehend the interplay between various traffic participants and make predictions and decisions based on continuous temporal data.

Image Source: Internet

This implies that models demand enhanced expressive capabilities, which in turn, generate greater computational demands.

However, a frequently overlooked aspect is that the number of model parameters does not directly correlate with the actual computational load. Parameters dictate the model's memory capacity, while computation determines its processing speed.

The on-board computational power required for a model also hinges on factors such as the number of input tokens, model architecture, inference methodology, data precision, and hardware-software integration.

Particularly for models like VLA that necessitate continuous video processing, the computational strain arises not only from the model itself but also from the escalating number of visual tokens.

Images captured by cameras, after visual encoding, generate a plethora of tokens.

If the number of cameras increases, resolution improves, or the historical video context extends, the volume of information the model must process during each inference also rises. Consequently, the computational pressure on intelligent driving models stems from both the model and the input—models are becoming increasingly complex, and the information fed into them is expanding. This is why merely augmenting chip processing power cannot fully resolve the issue.

02 Why Is Model Compression Still Essential as Processing Power Rises?

It is undeniable that on-board processing power continues to escalate.

BYD's unveiled 4nm intelligent driving chip, Xuanji A3, officially disclosed that three chips working in tandem can deliver a total processing power exceeding 2100 TOPS, while also prioritizing power consumption management and processing power utilization.

This indicates that the industry is not averse to augmenting on-board processing power but is seeking a new equilibrium between model expansion and processing power growth.

Automakers are no longer blindly pursuing the scale of on-board model parameters but are placing greater emphasis on the alignment between parameter scale, processing power consumption, and actual user experience.

Image Source: Internet

However, on-board processing power cannot be expanded indefinitely like data center processing power.

Augmenting on-board processing power introduces pressures in terms of chip cost, power consumption, heat dissipation, memory bandwidth, and system design. For mass-produced vehicles, the computing platform must also adhere to the requirements of long-term stable operation and large-scale deployment.

More crucially, increasing TOPS does not guarantee that all this processing power will translate into tangible model performance.

If the model harbors significant redundant computation or if data transfer, memory access, and operator execution efficiency are subpar, even if the chip provides higher peak processing power, it may not be fully leveraged.

Therefore, the genuine challenge on the vehicle side is not the availability of more processing power but whether the same tasks can be accomplished with less computation or more intricate tasks can be tackled with the same processing power.

This is where model compression truly shines.

03 Compression Targets Inefficient Computation, Not Capability

A clear distinction must be made here.

Model compression is not merely about excising a portion of parameters from a large model to yield a smaller one. Its true objective is to diminish unnecessary computation while preserving model capabilities as much as feasible.

Token compression serves as a prime example.

For continuous video input, not every visual token carries equal informational weight. Some tokens encompass key content such as road structure and traffic participants, while others may exhibit pronounced spatial or temporal redundancy.

If these redundant information fragments can be identified, the number of tokens that the subsequent model must process can be reduced. This approach does not imply that the model perceives less but rather that it concentrates its limited computational resources on more valuable information.

Image Source: Internet

In addition to token compression, techniques such as knowledge distillation, quantization, and model architecture optimization are also employed.

Distillation involves leveraging a more robust model to assist a smaller model in learning, enabling the transfer of certain capabilities to a model more amenable to on-board deployment.

Quantization reduces computational and storage pressure by lowering numerical precision.

Architecture optimization further modifies the internal computational methods of the model, rendering it more compatible with specific on-board chips.

These techniques address diverse issues but share a common objective: to allocate more computational resources to the components that genuinely influence intelligent driving capabilities under constrained on-board processing power.

04 The Crux Lies in How Models and Processing Power Evolve Together

After delving into these discussions, it becomes evident that the trends of increasingly larger intelligent driving models, elevated on-board processing power, and ongoing model compression are not three disparate phenomena. They actually constitute a cohesive technological chain.

The enhancement of model capabilities necessitates larger model capacities and more intricate computations; the expansion of models, in turn, propels the continuous upgrading of on-board processing power. However, merely augmenting processing power introduces engineering constraints such as power consumption, cost, and heat dissipation. Therefore, techniques like token compression, distillation, quantization, architecture optimization, and hardware-software synergy are indispensable to further enhance computational efficiency.

The ultimate outcome is not a one-way escalation where larger models lead to more potent chips, and more potent chips lead to even higher processing power.

Instead, a virtuous cycle is formed where model capabilities grow, on-board processing power follows suit, and computational efficiency improves concurrently, ultimately enabling more model capabilities to be effectively deployed on the vehicle side.

Image Source: Internet

This is also why the optimization of on-board AI is no longer solely a matter of the model itself.

Factors such as model design, the number of input tokens, which information fragments to retain, how the chip executes, how the compiler optimizes, and how memory data is transferred all ultimately influence whether an intelligent driving model can perform effectively on a vehicle.

Therefore, model compression is not at odds with large models.

The cloud aspires for more robust model capabilities, while the vehicle side aims to accommodate (chéngzài, meaning 'host' or 'support') more capabilities with limited computational resources.

What genuinely warrants attention in the future is how much truly valuable intelligence can be realized under the same on-board processing power.

#AutonomousDrivingLargeModels #ModelDistillation #ModelCompression