In 2025, the global market size for end-to-end neural network-based autonomous driving systems is estimated at approximately $671.9 million. By 2026, this figure is projected to grow to $741.5 million.

When expanding the scope to include end-to-end intelligent driving solutions, the global market size is expected to reach $10.035 billion by 2026 and is projected to grow to $87.359 billion by 2033.

Behind this rapid market expansion lies rapid technological iteration.

Over the past two years, end-to-end architectures have largely replaced traditional modular approaches. The industry's focus is now shifting from whether autonomous driving is possible to how well it can be performed. At the heart of this transformation is the technological leap from imitation learning to reinforcement learning.

01 Why Is Imitation Learning No Longer Sufficient?

In recent years, the dominant training method for end-to-end autonomous driving systems has been imitation learning. This approach involves having neural networks observe vast amounts of human driving data to learn how humans operate in specific scenarios.

Under this framework, traditionally separate modules such as perception, prediction, and planning are integrated into a unified neural network. The model directly learns driving strategies from sensor inputs like cameras.

Image Source: Internet

Tesla was the first to implement this approach with FSD V12, compressing the entire process from perception to control into a single neural network. From camera inputs to steering and braking commands, everything is handled by one model.

While effective, imitation learning quickly revealed its limitations. Its essence is replicating existing human driving behaviors, meaning the model's learned behavior distribution cannot exceed the range experienced by human drivers.

This leads to a critical issue: the safety-critical scenarios that autonomous driving must handle are precisely the extreme situations rarely encountered by human drivers.

Such scenarios occur infrequently on real roads, making data collection extremely costly. Yet, they directly determine the system's safety boundaries.

Even with massive amounts of conventional driving data, the model's generalization ability in long-tail scenarios remains uncertain.

Moreover, imitation learning's training objective is merely behavior replication, not optimization.

Real driving data contains numerous unsafe or uncomfortable operations. If the training goal is simply to fit the neural network to this data, the system will learn human drivers' flaws as well.

Additionally, distribution shifts between open-loop training and closed-loop reasoning affect model performance.

Models perform well on offline datasets but, when deployed in real-world dynamic environments, small errors accumulate frame by frame, eventually deviating from the intended trajectory.

This phenomenon, known as covariate shift, is one of the most challenging issues in applying imitation learning to autonomous driving.

02 How Does Reinforcement Learning Transform Training Logic?

Reinforcement learning's training logic differs fundamentally from imitation learning. Instead of mimicking behaviors, it sets goals like safely and efficiently completing driving tasks, then lets the model explore, receive feedback, and gradually optimize its behavior in an environment.

In autonomous driving, reinforcement learning primarily adopts deep reinforcement learning frameworks, combining deep neural networks with reinforcement learning.

Given driving's continuous action space (steering angle, throttle, braking, etc.) and high-dimensional state inputs (multi-sensor data), commonly used algorithms include Deep Deterministic Policy Gradient (DDPG), Soft Actor-Critic (SAC), and Proximal Policy Optimization (PPO).

Among these, PPO has gained the widest adoption due to its training stability and relatively high sample efficiency.

Reinforcement learning is essentially a continuous trial-and-error loop. The agent perceives environmental states and makes action decisions, the environment provides a reward signal, and the agent adjusts subsequent decisions based on this feedback.

In this process, reward function design is most critical. It must balance multiple dimensions such as safety, comfort, and traffic efficiency, making it highly challenging. The reward function directly defines which behaviors are encouraged and which should be avoided.

Image Source: Internet

In 2026, the application of reinforcement learning in end-to-end autonomous driving is accelerating.

Horizon Robotics' HSD V2.0, released in June, features core upgrades with world models and end-to-end reinforcement learning as dual technological foundations. NIO fully deployed its world model + closed-loop reinforcement learning architecture to hundreds of thousands of vehicles in January 2026. Momenta announced the mass production launch of its reinforcement learning world model at the Beijing Auto Show in April.

Additionally, Qualcomm AI Research's CLEAR framework enables large-scale closed-loop reinforcement learning training for end-to-end autonomous driving by learning residual waypoint strategies on top of pre-trained VLA policies.

CLEAR achieved new state-of-the-art results on benchmarks like CARLA longest6 v2 and Bench2Drive.

The World Engine framework, jointly proposed by Hong Kong University's Li Hongyang team, Huawei, and Tsinghua University, transforms real driving failure scenarios into closed-loop interactive simulation worlds and uses reinforcement learning for post-training. It achieved a 45.5% reduction in collision events during Huawei ADS's closed-loop validation.

03 World Models: The Training Ground for Reinforcement Learning?

For reinforcement learning to be effective, it requires an environment where models can repeatedly trial and error. Real roads obviously cannot permit this—no one would allow a vehicle to learn risk avoidance through trial and error on highways. This introduces the concept of world models.

World models simulate how the world might change in the next few seconds before making decisions.

For example, if a vehicle ahead is slowing down, world models can explore multiple possibilities—such as the vehicle coming to a complete stop or changing lanes—and evaluate what actions the ego vehicle should take in each scenario.

This "think before acting" capability provides the necessary training environment for reinforcement learning.

Image Source: Internet

In 2026, the combination of world models and reinforcement learning has become standard for leading intelligent driving companies. Intelligent Driving Frontier has also discussed this trend: Why are top intelligent driving players betting on reinforcement learning?

Horizon Robotics' HSD V2.0 features a world model + end-to-end reinforcement learning dual-engine architecture, enabling the intelligent driving system to autonomously trial and grow through reinforcement learning in a virtual physical world.

NVIDIA unveiled Cosmos 3 at GTC Taipei on June 1, 2026, positioning it as a unified physical AI foundation model for visual reasoning, world generation, and action generation.

At CVPR 2026, NVIDIA demonstrated a physical AI agent skill toolkit based on Cosmos 3 for tasks like scene reconstruction and closed-loop reinforcement learning.

Also at CVPR 2026, NIO had 12 papers accepted, four of which focused on world models. These papers addressed depth estimation, 3D reconstruction, traffic rule cognitive assessment, and safety risk prediction, covering key aspects of world models from low-level perception to high-level decision-making.

04 From Open-Loop to Closed-Loop: What Is the Industry Experiencing?

End-to-end autonomous driving is currently at a critical transition stage from open-loop imitation learning to closed-loop reinforcement learning.

Open-loop imitation learning offers high training efficiency and relatively simple implementation but has limited generalization ability. Closed-loop reinforcement learning allows models to continuously optimize through interaction but faces challenges like domain gaps between simulated and real environments and computational efficiency.

As a result, the current mainstream approach is a two-stage strategy: pre-training with imitation learning to establish basic driving capabilities, followed by fine-tuning with reinforcement learning.

NIO's latest technical architecture features a three-layer training framework: world models + supervised fine-tuning + closed-loop reinforcement learning.

Additionally, model scales are shifting from millions to billions of parameters; data volumes are upgrading from terabytes to petabytes; and training iteration cycles are compressing from monthly to daily.

These changes are redefining the upper limits of end-to-end model capabilities from a technological perspective, while market demand changes also illustrate this trend.

Image Source: Internet

In 2024, the penetration rate of Level 2 and Level 2+ intelligent driving was 60%, rising to 88% in 2025, and is expected to exceed 95% by 2026-2027.

This rapid increase means autonomous driving systems must maintain stable performance across broader scenarios, creating a more urgent need for training paradigm upgrades.

Meanwhile, the industry's technological paths continue to both diverge and converge. VLA (Vision-Language-Action) models introduce large language models as reasoning hubs in end-to-end frameworks, enabling systems to understand scenarios and make explainable decisions.

World models allow systems to simulate environmental evolution internally, predicting how the world will change after actions.

From 2025 to early 2026, debates between these two approaches were intense. However, by the second half of 2026, more companies realized that VLA and world models are not mutually exclusive.

One handles understanding what to do now, while the other predicts what will happen afterward. The two can be perfectly integrated into a single framework.

It is foreseeable that as reinforcement learning training frameworks mature and world model capabilities improve, the upper limits of end-to-end autonomous driving systems will continue to be pushed higher.

#AutonomousDriving #ImitationLearning #ReinforcementLearning #EndtoEndLargeModels