Produced by Zhineng Technology

AI can already write code, make videos, and answer complex questions—but the moment it needs to drive a car or operate a robot, things get far harder.

Construction cones that suddenly appear at an intersection, lane markings you can barely make out in heavy rain, forklifts that momentarily cut across a factory floor—each of these drags the intelligence of the digital world into the risks of the real one.

By 2026, NVIDIA's Cosmos, Omniverse, Alpamayo, and AlpaSim form a toolchain spanning data, simulation, and on-vehicle operation; Waymo, too, has publicly demonstrated how generative world models can reproduce rare and dangerous road situations.

These tools let AI try, fail, and be tested over and over in virtual scenarios before it ever enters reality. The whole point of physical AI is to carry actions learned in simulation reliably into the real world.

01 A chat AI can afford to be wrong; a driving AI must live with the consequences

You're driving through a familiar intersection. The navigation says go straight, but construction has suddenly appeared up ahead. A few cones block half a lane, and a worker stands at the roadside waving vehicles into the opposite lane. A car behind is urging you on, and an e-bike is preparing to squeeze through the gap.

An experienced driver slows down first, watches the hand signals, checks for oncoming traffic, and only then decides whether to proceed.

In 2026, physical AI is bringing the real world into computers, letting cars and robots rehearse repeatedly in the virtual world.

In recent years, large models have shown strong language and knowledge-processing capabilities. They can analyze images, write programs, generate videos—and even hand you a set of action plans that looks very well organized.

But once a machine starts moving, the answer becomes an action.

When a robot grasps a cup, too much force from its fingers may break it; if a robotic arm on a factory floor misjudges the position of a workpiece, it may hit the fixture; if an autonomous vehicle makes a judgment half a second late, the distance has already changed.

The physical world responds to AI's actions in turn. When an autonomous vehicle sees the car ahead suddenly merge, it chooses to swerve left. That steering changes its distance from the car behind, from oncoming vehicles, and from the median barrier. The actions of every other traffic participant may then change accordingly. The system must keep observing, re-judging, and correcting in time.

In the past, road data was accumulated by having test vehicles drive more roads and recording what happened. But dangerous events on the road are low-frequency and random, and reproducing them is costly. You can't arrange for a car to suddenly drive the wrong way at a real intersection every day just to train emergency avoidance.

A large amount of data has not solved all the problems. What is truly scarce are the scenarios automakers very much want the system to learn but are difficult to obtain safely in reality.

In February 2026, Waymo publicly demonstrated its generative world model, including scenarios such as wrong-way vehicles, abnormal weather, and flooded roads—letting the autonomous driving system face a batch of events that are hard to encounter naturally before it ever enters real roads.

The future capability of autonomous driving R&D will be half determined by how much data real vehicles have collected, and half by whether sufficiently difficult problems can be actively designed and verified.

It sounds a lot like a video game: draw the roads, buildings, cars, and pedestrians, then put an AI in to drive.

In the image it is raining heavily and the road looks wet, yet the vehicle still has the braking capability it would have on dry asphalt; the camera image has backlight, but there is no real glare or exposure change; when the robot picks up a box, the box's surface material, mass distribution, and friction are simplified.

Actions learned by AI in such an environment may fail in reality.

Digital twin: create an updatable digital file for real equipment.

Logistics passages, robot work areas, and production equipment locations in a factory can all be placed into a unified 3D scene to help engineers change layouts and coordinate.

World model: let AI predict what will happen next.

If the car turns left, how might surrounding vehicles change? If the robotic arm releases its gripper, will the object slip? The world model generates possible subsequent states based on the current state and actions.

Closed-loop simulation: make every action produce a consequence, then feed the consequence back to the AI.

The system brakes, and the virtual vehicle slows down accordingly; the car behind may react; the AI receives new images and sensor information and decides what to do next.

Only when connected together do they form a useful training environment: scenarios corresponding to reality, a model capable of predicting changes, and a testing process that can continuously provide feedback.

The key is the closed loop.

If you only replay a driving video, no matter whether the AI chooses to brake or steer, the pedestrian in the video will walk along the predetermined trajectory. This is like looking at exam answers; the system can hardly know what consequences another action would have caused.

Closed-loop simulation allows AI to try different actions and then observe how the environment changes. In R&D experience publicly shared by Waymo in 2026, this kind of causal feedback was particularly emphasized.

02 How autonomous driving iterates

NVIDIA has formed three interconnected computing stages around autonomous driving: cloud training, simulation and validation, and vehicle-side inference.

◎ The first part is in the data center.

Data collected by vehicles is filtered and annotated here, and models also undergo large-scale training here. In the past, the system mainly learned "how humans usually drive when encountering this kind of scene"; now, large models are also used to understand the intentions of traffic participants, explain action choices, and predict future trajectories.

◎ The second part takes place in virtual scenarios.

NVIDIA Omniverse participates in building and running simulation environments, the Cosmos world model is used to generate and deduce scenarios, and AlpaSim and AlpaGym serve closed-loop testing and training for autonomous driving.

Developers can reconstruct an intersection from real road data, then change the weather, vehicle behavior, and obstacle positions to specifically test the system's weak points.

◎ The third part returns to the vehicle.

Production vehicles have constraints on power consumption, heat dissipation, cost, and response time, so it is impossible to copy data center computing directly onto the vehicle.

NVIDIA's Alpamayo 2 Super, launched in 2026, is a 34-billion-parameter reasoning-oriented vision-language-action model.

One of its public positions is to undertake the tasks of larger models during the R&D phase, and then transfer capabilities through knowledge distillation to smaller models suitable for vehicle-side operation.

How smart the cloud model is and whether the vehicle-side system can act promptly and stably are two separate exams.

Competition in AI cars has long extended beyond chips: who can discover model blind spots faster, who can turn problems into reproducible tests, and who can make vehicle-side software and hardware respond in time will determine the final experience.

Compared with open roads, the physical environment of a factory is easier to describe and constrain.

Production line positions are fixed, the robot's task scope can be defined, and logistics routes, sensors, and equipment status are also easy to connect to management systems. The first batch of digital twin applications in industry began with planning, commissioning, and troubleshooting.

BMW used Omniverse-related development tools to build the FactoryExplorer digital twin application, supporting engineers in collaboratively viewing and optimizing factory layouts and carrying out robot and vision AI-related development before on-site deployment.

Suppose a production line needs to add several transport robots.

The traditional approach requires coordinating space, equipment, and personnel, and then conducting on-site commissioning; verifying routes, checking intersections, and adjusting work areas in a digital factory first can expose a large number of conflicts in advance.

After the robots enter the factory, real-world issues such as sensor errors, tire slippage, fixture tolerances, and temporary entry by personnel still have to be handled. Simulation filters out many errors, but on-site acceptance and safety protection still cannot be omitted.

In industry, what this technology changes first is the way engineers work: problems that used to be debugged item by item after equipment was installed are beginning to shift forward to the design stage.

Autonomous driving is much harder. Streets are open environments, road conditions and human behavior are difficult to constrain, and external participants will keep changing the scenario.

Factories and robotaxis are competing for the same capability: making AI trial and error happen earlier and making the cost in reality as small as possible.

If a scenario generator has never correctly simulated slippery roads, then no matter how many rainy-day videos it generates, it cannot teach the real braking boundary.

Digital humans can walk and ride bicycles, but their movements lack the hesitation and temporary changes of direction of real people. The autonomous driving system will still be trained with incorrect expectations, and safety capability must be verified through repeatable testing and real-world performance.

The National Highway Traffic Safety Administration (NHTSA)'s autonomous driving research adopts simulation, test tracks, and road testing at the same time. Only by combining several methods can the system's capabilities and boundaries be fully observed.

Summary

Physical AI will next have to face roads, factories, warehouses, and homes: see clearly what is happening around it, predict what will happen next, and then reliably execute actions.

This will give rise to a long industrial chain: sensors for collecting data, software for building scenarios, GPUs for training models, chips for vehicle and robot sides, and verification and safety engineering.