In the realm of autonomous driving, the planning system functions as the decision-making core, akin to the human brain, following the acquisition of environmental awareness. The perception module translates sensor data—captured by cameras, radars, and lidars—into actionable information. Meanwhile, the localization module pinpoints the vehicle's precise position, and the prediction module evaluates potential changes in the behavior of surrounding traffic entities. Together, these components feed into the planning system, which addresses two fundamental questions: the optimal route for the vehicle and the appropriate speed and posture for traversal.

Planning extends beyond strategic decision-making—such as lane changes, deceleration, or waiting—to the refinement of these strategies into executable trajectories and speed profiles. These are then relayed to the controller for implementation. Crafting an effective planning system necessitates a delicate balance between safety, comfort, and efficiency, all while adhering to traffic regulations and responding reliably to unforeseen circumstances.

What Inputs Does Planning Rely On, and How Does It Manage Uncertainties?

The planning system's decisions are not made in a vacuum; they hinge on both external and internal state information. Localization and mapping provide semantic data, including road network layouts, lane details, speed limits, and intersection specifics, which are crucial for navigating complex intersections and long-distance journeys. Perception relays information about the positions, velocities, and orientations of nearby vehicles, pedestrians, and obstacles to the autonomous driving system. However, perception is not infallible and may suffer from delays, false positives, or missed detections. To mitigate these uncertainties, the autonomous driving system employs fusion and filtering techniques at the input stage to stabilize state estimation. The prediction module typically generates multiple possible outcomes or probability distributions, rather than a single deterministic future trajectory. When utilizing these predictions, the planning system considers their confidence levels, treating high-risk scenarios as conditions warranting more cautious handling. Furthermore, the vehicle's dynamic constraints, steering capabilities, and energy status limit the feasible trajectories. Planning must ensure that its outputs are not only theoretically viable but also practically executable by the vehicle.

Common Technical Approaches for Decision-Making and Trajectory Generation

Planning is generally categorized into two levels: the behavioral layer and the motion layer. The behavioral layer is tasked with abstract decision-making, such as determining whether to change lanes, yield to pedestrians at intersections, or overtake. The motion layer, on the other hand, translates these decisions into specific trajectories and speed profiles, taking into account the vehicle's dynamic constraints.

The behavioral layer may utilize highly interpretable rules or state machines, which are easier to validate and comply with regulatory standards. However, as scenario complexity increases, the number of rules can proliferate rapidly. In recent years, learning methods have been employed to supplement or generate behavioral candidates, enhancing flexibility in common scenarios. Nevertheless, due to concerns regarding verifiability, learning modules are often used as suggesters, with final decisions still made by controllable optimizers or rule-based layers.

The motion layer employs a variety of implementation methods, including search-based route planning for global navigation, sampling techniques for local obstacle avoidance, and optimization-based continuous trajectory generation. Optimization-based methods can uniformly model comfort, collision constraints, and dynamic limitations, yielding smooth and coherent trajectories by solving for optimal values. Model predictive control, which is capable of solving constrained motion problems over a future time horizon online, is frequently used for joint speed and steering operations, bringing vehicle execution closer to the ideal state. In practical applications, several methods are often combined, using rapid sampling to generate candidate trajectories and then refining the selection with higher-quality optimizers to ensure real-time performance while enhancing planning quality.

Safety, Robustness, and Collaboration with Other Modules

For the planning system, simply aggregating algorithms is far from adequate. A truly industrially viable planning system must incorporate safety, verifiability, and real-time performance at the engineering level. Planning systems often employ multiple layers of protection: the primary layer consists of regular planning outputs; the secondary layer is an online avoider based on collision detection and safety buffers; the tertiary layer is the Minimal Risk Maneuver, enabling the vehicle to dock or decelerate to a safe state in a controlled manner when perception is compromised or uncontrollable risks are detected. Collision detection typically relies on geometric models (such as bounding boxes and circular approximations) combined with time-dimensional occupancy predictions, performing rapid reachability analysis or conservative safety distance estimations online.

In terms of robustness, considerations include sensor noise, frame drops, map discrepancies, weather impacts, and long-tail extreme scenarios. Strategies to cope include integrating probabilistic methods (such as POMDP) or distributed robust planning into the decision-making chain, employing more conservative cost functions, or explicitly considering prediction uncertainties in planning. Additionally, online replanning and backtracking mechanisms are essential. When the environment changes, the planner must regenerate trajectories with minimal delay while ensuring continuity during switching to avoid abrupt movements.

For autonomous vehicles, real-time performance is paramount. Planning typically operates in cycles ranging from tens to hundreds of milliseconds, with specific frequencies depending on the level. Local trajectory generation usually cycles at the 20 ms–100 ms level, while behavioral decisions may range from hundreds of milliseconds to seconds. Due to limited computational resources, many systems adopt a hierarchical architecture and make approximations to reduce online computation. For instance, they may first use rapid sampling to obtain candidate trajectories and then run high-quality optimizers on the candidate set to refine the optimal solution. Furthermore, simulation and large-scale scenario replay are indispensable verification tools, covering long-tail conditions through millions of kilometers of simulation and scoring and improving planning strategies.

Final Words

Planning is not an isolated module; it requires seamless collaboration with prediction, perception, and control. The prediction module provides future trajectory distributions of other road users, which the planning system uses to make safe and efficient choices. Perception is responsible for promptly detecting obstacles and feeding information into the planning pipeline, with perception delays or errors directly affecting the conservativeness of planning strategies. The controller must accurately execute the planned trajectories, with control tracking errors fed back into the planning constraints (e.g., planning with control error margins). The planning system in autonomous driving is a synthesis of algorithms, control theory, probabilistic methods, and engineering implementation details. To achieve safety, comfort, and comprehensive coverage of real-world complex scenarios, relying on a single technology is insufficient. Multiple methods must be integrated and continuously refined through extensive simulation and road testing.

-- END --