Imagine this scenario: An autonomous vehicle is traveling at 110 km/h on a highway when a cardboard box suddenly falls off a truck ahead. The box is not present in any standard object annotation dataset. How would the autonomous driving system respond? A traditional perception system would likely classify the box as unknown, then ignore or mishandle it, ultimately causing the vehicle to collide directly with it.

This solution, which is precise with known categories but blind to unknown objects, is a long-standing issue with traditional perception systems. In real-world traffic scenarios, irregular objects on the road, temporarily stacked construction materials, or even a large piece of plastic blown by the wind could pose long-tail safety risks. The introduction of the Occupancy Network (ON) offers a different approach to this problem, leading many to speculate boldly: With ON, is LiDAR still necessary?

When Perception Shifts from Guessing Objects to Recognizing Space

Traditional perception methods based on 3D object detection essentially simplify the three-dimensional world into a list of objects, with each object represented by an axis-aligned bounding box. The box contains the object's position, dimensions, and orientation, while its label indicates the object's category. This representation is effective in most scenarios but has two inherent flaws.

The first flaw is geometric distortion. A bounding box includes significant empty space—for example, the irregular gap between a truck's cargo area and cab. The box marks the entire region as occupied, leading the planning module to perceive obstacles where none exist. Conversely, if an object has an irregular shape, such as a long arm extending from a construction vehicle or a fallen tree, a square box may fail to fully cover the object's actual occupied area, creating collision risks.

The second flaw is its inability to recognize unannotated objects. If a traffic cone is present in the training dataset, the model can identify it; if a cardboard box is absent, the model treats it as background. However, the real world contains an almost infinite variety of objects, and no dataset can exhaustively cover all possibilities.

Image Source: Internet

Instead of guessing objects, the Occupancy Network addresses a more fundamental question: Is each tiny unit of 3D space occupied? It discretizes space into numerous regular voxels. For each voxel, the network predicts only two things: whether it is occupied and, if so, which category it belongs to. Category prediction is generally optional—in practice, only a few critical categories like cars and pedestrians are subdivided, while the rest are uniformly labeled as generic obstacles. This representation does not rely on prior knowledge of object shapes; as long as something exists at a location, regardless of its shape or whether it has been seen before, it is marked as occupied. The principle of prioritizing geometry over semantics is the source of its ability to handle open-world scenarios.

The Occupancy Network typically takes surround-view camera images as input. After feature extraction and perspective transformation (e.g., spatial cross-attention in BEVFormer), it generates a Bird's Eye View (BEV) feature map. This map is then projected upward into 3D space via 3D convolution or a Transformer decoder, ultimately outputting an occupancy probability tensor of size (H, W, Z).

For Occupancy Networks, output resolution is a critical parameter. In current mainstream solutions, voxel sizes typically range from 0.2 meters to 0.5 meters. Finer voxels enhance perception of small objects but increase computational load cubically. A space covering 80m × 80m × 6m in front of the vehicle with 0.2m voxels contains 4.8 million voxels, imposing high demands on chip computing power and memory bandwidth.

Image Source: Internet

The integration of temporal information is another significant enhancement for Occupancy Networks. Early versions processed only single-frame images, outputting static occupancy at the current moment. However, autonomous driving requires perceiving motion, such as determining whether a suddenly braking vehicle ahead will collide with the ego vehicle. To address this, 4D Occupancy Networks introduce an additional prediction head to output the flow field for each voxel—its motion speed and direction between adjacent frames. By fusing historical information from multiple consecutive frames, the network establishes spatiotemporal consistency for dynamic objects, avoiding flickering artifacts where objects appear and disappear abruptly.

Additionally, researchers have begun coupling Occupancy Networks more tightly with planning modules. Traditional approaches follow a perception-prediction-planning pipeline, with the network's output handed off to downstream modules. Between 2025 and 2026, the emergence of Occupancy World Models attempts to have the network directly output occupancy predictions several seconds into the future, forming an end-to-end differentiable framework. For example, the CascadeOcc framework incorporates scene intrinsic structure into the network through a hierarchical structure, enabling prediction of future occupancy states and direct generation of driving trajectories. This perception-as-prediction approach reduces information loss in intermediate steps and enables faster system responses.

From Technical Routes to Industrial Implementation: Diverse Approaches

Companies and technical teams have pursued differentiated paths in exploring Occupancy Network technology. Tesla was the first to publicly propose Occupancy Networks and deploy them in production vehicles. At its 2022 AI Day, Tesla demonstrated a vision-based Occupancy Network capable of real-time 3D space reconstruction around the vehicle, including curbs, traffic cones, and various irregularly shaped obstacles. Tesla's approach relies solely on cameras and powerful neural networks to infer voxel occupancy states, premise (prerequisite) being access to massive amounts of high-quality training data—Tesla's fleet collects millions of kilometers of real-world driving video daily. By 2023, Tesla further shifted to an end-to-end architecture, integrating perception, prediction, and planning into a single neural network, reducing code volume from tens of thousands to a few thousand lines.

Domestic automakers have adopted more diversified strategies. NIO's Occupancy Network primarily reduces false detection rates for irregular obstacles, especially in urban environments with non-standard objects like construction barriers and temporary roadblocks, preventing dangerous decisions due to recognition failures.

XPENG's XNet architecture supports pure vision-based mapless driving. According to XPENG's May 2024 AI DAY announcement, XNet achieves end-to-end intelligent driving by aggregating dynamic XNet, static XNet, and the industry's first mass-produced pure vision 2K Occupancy Network. It perceives an area 1.8 times the size of a football field and accurately identifies over 50 target objects.

Image Source: Internet

Huawei integrates occupancy capabilities into its ADS system while retaining LiDAR as a supplement to visual networks. The core of Huawei ADS 2.0 is the GOD network and OCC Occupancy Network, essentially fusing LiDAR and camera data rather than relying solely on one sensor type. In high-end models, LiDAR remains as a safety redundancy, forming a vision-primary, LiDAR-secondary architecture.

Notably, OccNet (Scene as Occupancy), jointly proposed by SenseTime and the Shanghai Artificial Intelligence Laboratory, employs a cascaded temporal voxel decoder to reconstruct 3D occupancy fields. Its core explores occupancy perception capabilities within a pure vision framework. Experimental results in the paper show that applying OccNet to motion planning reduces collision rates by 15% to 58%, a quantifiable improvement.

How Is LiDAR's Role Evolving?

As Occupancy Networks grow more capable, an online discussion has emerged: Is LiDAR still necessary with Occupancy Networks?

Pure vision-based Occupancy Networks demonstrate impressive geometric perception abilities but have clear limitations. Camera imaging relies on ambient lighting, making image sensors' dynamic range insufficient to retain details in both highlights and shadows during nighttime, tunnel entrances/exits, and scenes with direct strong light. In dark environments where human vision struggles, pure vision systems' reliability declines further. Additionally, rain, fog, and snow reduce image contrast, making distant obstacle detection difficult. These scenarios represent persistent challenges for pure vision solutions, whereas LiDAR, as an active sensor, does not rely on ambient light and can still output point cloud data under these conditions.

Image Source: Internet

When LiDAR was first discussed, many people's first reaction was its high cost. Around 2020, automotive-grade LiDAR prices were in the tens of thousands of dollars, but by 2025, entry-level products had dropped to 2,000–3,000 RMB. While prices fell, performance improved. Huawei's 896-line image-grade LiDAR achieves near-image-level resolution, meaning LiDAR can now output not just geometric information but also perceive object material and texture to some extent. Hesai's sixth-generation chip platform further enables simultaneous 3D space perception and object color information collection on a single chip. Lighter, smaller, cheaper, and more powerful LiDAR is transforming its positioning (positioning) as an expensive accessory.

LiDAR remains irreplaceable in data loops. Nearly all advanced vision-based Occupancy Networks rely on LiDAR-generated point cloud data to provide ground truth labels during training. A collection vehicle equipped with LiDAR can precisely determine whether each voxel is occupied, with these data processed into learning targets for visual networks. In other words, while LiDAR may no longer be mandatory in vehicle perception systems, it remains a crucial tool for training and iterating visual models.

Image Source: Internet

Current industry choices confirm that Occupancy Networks and LiDAR coexist rather than replace each other. According to Gasgoo Automotive Research Institute data, total LiDAR installations in China's passenger vehicle market reached 2.756 million units in 2025, with a 21% penetration rate in new energy passenger vehicles. High-Tech Intelligent Vehicle Research Institute data shows that LiDAR shipments for passenger vehicles in China reached 3.2484 million units in 2025, a 112.07% YoY increase.

While the two data sources use slightly different metrics, both point to the same trend: LiDAR is rapidly becoming widespread. Meanwhile, nearly 100% of L3 and higher-level intelligent driving models are equipped with LiDAR, reflecting the industry's emphasis on perception redundancy. Some automakers adhere to pure vision routes, using algorithms and data to compensate for sensor limitations; more automakers make LiDAR standard equipment, especially for models facing complex urban environments where LiDAR is indispensable.

From Replacement to Symbiosis: The True Landscape of Technical Routes Emerges

Combining the above analysis, a clear judgment emerges: Occupancy Networks will not render LiDAR obsolete. Instead, their relationship is shifting toward a more complex collaborative dynamic.

From a quantitative safety perspective, academic research confirms that planning based on Occupancy Networks can reduce collision rates by 15% to 58%. This improvement comes from decision-making gains enabled by voxel-level precise (fine-grained) perception—vehicles no longer need to identify, classify, and avoid obstacles but can directly conclude that certain areas are impassable. Meanwhile, vehicles equipped with LiDAR show significantly lower intervention rates in accidents involving nighttime and severe weather, indicating that multi-sensor fusion remains irreplaceable in extreme scenarios.

In the future, autonomous driving perception systems will likely adopt a tiered configuration. In cost-sensitive market segments, pure vision-based Occupancy Network solutions may become mainstream due to their hardware cost and algorithmic efficiency, meeting basic safety requirements. In applications demanding higher safety redundancy, LiDAR will still play a critical role, but its form may change. LiDAR may no longer operate independently as the primary perception source but instead deeply integrate with visual Occupancy Networks, forming a mutually verifying, redundant structure.

-- THE END --