When an intelligent driving vehicle encounters a problem on the road, the system records the relevant data, transmits it back to the cloud, and—after screening, labeling, and training—ultimately improves driving performance through model updates.

On paper, this process already constitutes a complete data closed loop.

But once the data comes back, are the problems really identified? Once the samples are labeled, does the model really learn? Once the model is trained, will the vehicle's real-road performance necessarily improve?

In some intelligent driving R&D workflows, data collection, transmission, and training have reached scale, yet the chain from problem discovery to capability validation still has breakpoints.

Data keeps growing, and the model keeps iterating—yet certain problems may recur.

This raises a question worth asking: is the data closed loop actually driving system capability improvement, or has it merely completed data collection and circulation?

01 Does More Transmitted Data Mean More Value?

For intelligent driving systems, data collection is not just about recording the vehicle's driving process.

Depending on the vehicle model and the sensor configuration of the intelligent driving system, the vehicle can collect camera images, LiDAR point clouds, millimeter-wave radar information, as well as data such as vehicle speed, steering angle, braking status, and system operating status.

This data can help R&D personnel reconstruct the road environment the vehicle was in at the time, the state of traffic participants, and the perception and decision-making process of the intelligent driving system.

However, the volume of data generated by vehicles every day is enormous. Not all data needs to be uploaded—let alone does all data have equal value.

What truly affects model capability is often the difficult scenarios that expose the system's weaknesses.

Consider a vehicle passing through a construction section: the original lane lines intersect with temporary markings, and surrounding vehicles keep cutting in. The system may exhibit unreasonable deceleration, hesitant lane changes, or abnormal trajectory planning.

Image source: Internet

Such scenarios deserve more attention from R&D personnel than straightforward driving on normal roads.

Therefore, intelligent driving companies need to establish a data screening mechanism.

Common data screening signals include driver takeover, abnormal system status, unstable perception results, deviation between predicted trajectories and actual motion, and abnormal planned trajectories.

Companies can trigger data recording or upload based on these signals, and then screen out truly valuable problem samples through cloud-based scenario mining and manual analysis.

However, there is an easily overlooked problem here: a trigger signal does not equal an actual problem.

Driver takeover may stem from system misjudgment, from the driver being unaccustomed to the system's driving style, or from the driver actively taking over out of personal driving preference.

Conversely, the absence of takeover does not necessarily mean the system's behavior is reasonable. In some cases, although the system makes no obvious errors, it has already exhibited unnecessary deceleration, excessive conservatism, or unnatural decisions.

If companies screen data only by number of takeovers, hard braking counts, or abnormal trigger counts, they may miss problems that do not trigger obvious alerts but do affect the actual driving experience.

In addition, the data collection mechanism itself has coverage bias.

If the system is more likely to record anomalies that have already been identified but cannot discover risks it has not yet perceived, then the dataset may continue to concentrate on known problems and have insufficient coverage of unknown scenarios.

Therefore, the first step of the data closed loop should not be merely to transmit as much data as possible, but to establish a mechanism that can discover problems, distinguish problems, and screen problems.

Otherwise, as data scale continues to expand, the effective samples that can truly drive system improvement may not increase in sync.

02 The Data Has Already Been Labeled—So Why Might the Model Still Fail to Learn?

After data screening is completed, companies still need to process the samples to clarify what scenarios they contain, what state the traffic participants are in, and what problem the intelligent driving system actually had.

This involves automatic labeling, manual labeling, and problem analysis.

Taking a construction section as an example, R&D personnel not only need to identify cones, barriers, vehicles, and lane lines, but also need to combine the road structure to determine the currently passable area and analyze why the vehicle did not choose a suitable driving path.

If only obstacle recognition and bounding box labeling are completed, without further analysis of the system's abnormal behavior, then although this data can enter the training process, it may not solve the original problem.

Because abnormalities in intelligent driving systems do not always come from perception errors.

For example, in a construction section, the perception system has already identified cones and temporary lane lines, but the trajectory generated by the planning module is too conservative and does not fully utilize the currently passable space—so the vehicle may still exhibit unnecessary deceleration or detouring.

If R&D personnel attribute the problem only to insufficient perception accuracy and then add labeled samples of related obstacles, the model may not achieve the expected improvement.

For traditional modular intelligent driving systems, R&D personnel can combine the outputs of perception, prediction, and planning modules to gradually locate where the problem occurs.

But in end-to-end intelligent driving systems, perception, prediction, and planning capabilities may be jointly completed by a unified model, and some intermediate processes are not easy to directly observe and explain.

At this point, problem analysis cannot rely only on the results of a single module. Instead, it needs comprehensive analysis combining model inputs and outputs, obtainable intermediate features, training samples, and actual vehicle behavior—and, when necessary, troubleshooting through controlled experiments and ablation analysis.

Image source: Internet

In addition, the consistency of data labeling is also very important.

For the same construction scenario, if different labelers have obvious differences in judging temporary lane lines, passable areas, or reasonable driving trajectories, label noise may be introduced into the training data, reducing data consistency and affecting the model's learning effect on related scenarios.

For some end-to-end intelligent driving models that use supervised learning, the training process needs to use human driving behavior, planned trajectories, or other forms of supervision signals.

If these signals themselves are biased, the model may learn unreasonable driving behavior.

Therefore, the value of data labeling is not just to attach labels to objects in the scenario, but more importantly to help R&D personnel understand why the system has a problem and how the training data should reflect reasonable driving behavior.

If the problem attribution itself is inaccurate, then subsequent training may very likely just repeatedly use a batch of data that does not truly solve the problem.

03 The Model Keeps Training—Why Do Problems Still Recur?

After data is screened and labeled, it enters the model training stage.

Many companies continue to increase the scale of training data, expand coverage of difficult scenarios, and improve system performance through model iteration.

But an increase in data volume does not necessarily mean the model can gain stronger generalization ability.

There is an important reason here: the model may repeatedly encounter similar scenarios during training without truly learning to cope with new scenario variations.

For example, some companies collect a large amount of construction road data, including different roads, different weather, and different traffic participants.

But if these samples mainly come from the same batch of roads and similar construction layouts, the model may perform well in familiar scenarios while still showing unstable judgment when switched to construction sections with different road structures and traffic conditions.

This involves the representativeness of training data, as well as the way training, validation, and test sets are divided.

In continuous video data, adjacent frames often have very high similarity.

If adjacent frames from the same driving video are randomly assigned to the training set and test set, the test set may contain a large number of near-identical images that the model has already seen during training.

This kind of data leakage makes test results look better, but it cannot truly reflect the model's performance when facing unseen scenarios. Therefore, dataset division cannot rely only on simple random sampling—it needs to be designed according to the evaluation objective.

Image source: Internet

If the goal is to validate the model's generalization ability to new roads, isolation can be done by geographic location; if the goal is to avoid near-identical samples from continuous video being distributed across sets, division by driving segment or collection sequence should be considered.

This does not mean similar scenario data is worthless.

On the contrary, adding samples for similar problems helps improve the model's ability to handle related scenarios.

But if there is a lack of reasonable isolation between training and testing, it will be difficult for companies to determine whether the model is learning transferable driving capability or merely achieving good performance within a familiar data distribution.

Another problem is that model training cannot automatically solve all problems.

When newly added data differs in distribution from the original data, the model's capability may fluctuate; when training samples are overly concentrated in a certain type of scenario, the model's handling of other scenarios may also be affected.

Even if the performance of a certain problem on the training set improves, it does not directly prove that the vehicle has already possessed stable coping capability on real roads.

Therefore, model training needs to focus not only on data scale and training metrics, but also on differences among samples, whether dataset division is reasonable, and the model's performance in unseen scenarios.

04 After the Model Update, Is the Closed Loop Really Completed?

After the data is processed and the model training is completed, the next step is to verify whether the improvement is effective.

But better performance in offline tests does not mean it will necessarily perform better in real traffic environments.

A common validation method is historical data replay.

R&D personnel input previously collected road data into the model and observe the model's perception, prediction, and planning results in the same scenarios. It should be noted that historical data replay is not naturally equivalent to open-loop testing.

If the system input remains fixed to historical records and is not affected by the new model's output, then it is open-loop replay; if the simulation environment can regenerate subsequent scenarios based on the ego vehicle's new decisions and motion state, then a closed-loop resimulation can be formed.

The difference between the two lies in whether the vehicle's behavior affects subsequent environmental changes.

In open-loop replay, the movement trajectories of historical traffic participants are already fixed. Even if the new model generates different driving actions, other vehicles still move according to the original records.

This method is suitable for comparing differences in the model's output on the same segment of historical data, but it cannot fully reflect the subsequent traffic changes that different driving decisions may bring.

Image source: Internet

Closed-loop simulation, on the other hand, allows the vehicle to interact with the simulation environment.

For example, when the intelligent driving vehicle is preparing to change lanes, other vehicles in the simulation environment can adjust their speed or position according to the ego vehicle's actions, thereby forming a new traffic state.

However, the effect of closed-loop simulation depends on the accuracy of environment modeling and traffic participant behavior models. If other vehicles still move according to preset trajectories and do not respond to the intelligent driving vehicle's behavior, then the simulation environment's restoration of real traffic interaction is still limited.

Therefore, simulation testing can help companies discover problems and compare solutions, but it cannot completely replace real-road validation.

In actual R&D, companies also need to combine regression testing, closed-course testing, and road testing to check whether model updates introduce new problems and whether the original problems have been solved.

For example, a model update may improve detouring capability on construction roads, but cause the vehicle to become too aggressive when meeting vehicles on narrow roads.

Validating only the performance of newly added scenarios without regression testing on existing scenarios may introduce another problem while solving one problem.

Model deployment and subsequent vehicle version updates also need to be included in a complete validation process.

Only when the improvement has been validated in the test environment and on real roads, and has not obviously harmed performance in other scenarios, can it be shown that this iteration has achieved actual results.

If, after the model update, there is no continued observation of vehicle performance, collection of new problems, and the next round of analysis, then the data closed loop is actually still stuck at the training and deployment stages.

05 What the data closed loop truly competes on is no longer just data scale

From data collection, scenario screening, and labeling analysis to model training, simulation validation, and real-vehicle testing, the data closed loop is not a single independent stage, but a continuously operating R&D process.

Data scale is of course important.

More real-road data can expand scenario coverage and provide richer samples for model training. But data scale cannot be directly equated with intelligent driving capability.

What truly affects data value is whether companies can identify meaningful problems from massive data, accurately find the causes of problems, transform problems into effective training samples, and confirm through validation that the model has indeed improved.

This also explains why different companies may still differ in the intelligent driving capability they ultimately develop even if they have similar data collection scale.

Some companies can transform abnormal behavior on the road into clear R&D tasks and continuously solve problems through data analysis, model training, and validation testing;

Image source: Internet

some companies, although they have accumulated a large amount of driving data, may have breakpoints in problem screening, labeling attribution, and validation, resulting in continuous data growth but insufficiently obvious actual improvement.

Data scale determines how many real scenarios a company can access, and data closed-loop efficiency determines how much capability improvement a company can gain from these scenarios.

For the intelligent driving industry, competition in the data closed loop is extending from data collection capability to the overall efficiency of problem discovery, cause analysis, model iteration, and engineering validation.

If these stages are not connected, then no matter how large the data scale is, it may just be a continuously expanding data warehouse.

What the data closed loop truly needs to close is not the transmission path of data from the vehicle side to the cloud, but the complete chain from real problems to engineering improvement. #AutonomousDriving #DataClosedLoop #ModelTraining