When we gaze at a photograph, our eyes can instinctively gauge the distance of objects within the image. This innate perception of space and distance is a skill humans develop from a young age.

For autonomous vehicles, possessing a similar capability is crucial for accurately interpreting road conditions.

What Is Depth Estimation?

Autonomous vehicles must ascertain the distances of objects in their surroundings. They need to swiftly determine whether an object ahead is a pedestrian or another vehicle, and precisely gauge whether that vehicle is ten meters away or between one to two hundred meters.

Depth estimation enables machines to estimate the distance of objects from themselves based on perceived images or sensor data, providing the computer with a comprehensible sense of 'space'.

In the realm of computer vision, this capability is known as Depth Estimation and forms a fundamental component of autonomous driving perception systems.

The outcome of depth estimation is depicted as a 'depth map'. Unlike a conventional photograph where each pixel represents color, in a depth map, each pixel signifies the real-world depth value corresponding to that pixel point. In simpler terms, it indicates the distance of objects in the image from the observer.

With a depth map, the onboard system can transform a two-dimensional image into a perception of three-dimensional space, which is vital for tasks such as path planning, obstacle avoidance, and speed control.

Why Is Depth Estimation Necessary for Autonomous Driving?

Merely providing an autonomous driving system with a photograph does not enable it to directly judge distances. This contrasts with humans' innate ability to perceive depth in photos; machines only perceive numbers and pixels.

Without depth information, computers can only ascertain the approximate shape, color, and category of objects, but not their actual spatial positions.

For instance, a vehicle may appear large and clear, but determining whether it is ten meters or one hundred meters away requires depth information.

Traditional depth perception methods employ sensing hardware like LiDAR, which directly measures distances using lasers and yields excellent imaging results. Consequently, many current autonomous driving systems depend on LiDAR to acquire depth information.

However, LiDAR is expensive, demands high computational power, and entails various subsequent issues such as installation and maintenance.

Depth estimation, as a computer vision technique, aims to utilize cost-effective cameras and algorithms to supplement or replace some of the expensive sensing hardware.

In other words, depth estimation technology empowers autonomous vehicles to predict distances from ordinary images captured by cameras.

For example, if there is a pedestrian ahead, the machine not only needs to recognize it as a person but also determine how many meters away that person is from the vehicle. This is the data furnished by depth estimation.

Without such three-dimensional perception, formulating safe driving strategies would be impossible, even if object categories could be identified.

How Is Depth Estimation Achieved?

Depth estimation fundamentally involves inferring spatial distances from images. Since a single image does not contain true depth information, this process necessitates a complex workflow.

Inferring distances in three-dimensional space from flat pixels and colors alone is a classic 'ill-posed problem'. Machines cannot ascertain true distances from just one image and must rely on geometric principles, prior knowledge, and vast amounts of data to aid in inference.

Currently, mainstream depth estimation methods can be categorized into two types.

One is the multi-view method, which involves observing the same scene simultaneously with two or more cameras from different perspectives. Traditional stereo vision algorithms are then employed to match and calculate disparity (i.e., determining the pixel offset of the same object from different perspectives) and convert it into depth information.

This is analogous to how our two eyes perceive stereoscopic images; our left and right eyes perceive slightly different images, and through this disparity, our brains can judge depth.

A similar principle can be applied in autonomous driving systems, where two cameras can achieve simple depth estimation.

Another more prevalent method is monocular depth estimation, which utilizes only one camera.

Since a single image does not contain disparity information, computers can still glean some inherent depth cues from images through extensive data and deep learning model training.

Visual signals related to depth, such as blurring of road surfaces, object shrinkage, and occlusion relationships, can serve as cues.

Deep learning models encode these cues through convolutional neural networks, feature extraction, and other techniques, then predict the depth of each pixel.

Monocular depth estimation presents certain technical challenges. The scale of real-world objects varies significantly, and the same pixels may correspond to vastly different distances in different scenes. Therefore, algorithms need to be trained on large-scale annotated data to enable the model to learn general depth patterns.

This process is akin to having the model read books; through tens of thousands of images with depth annotations, it comprehends what depth distribution corresponds to each visual feature.

The trained model can then provide reasonable depth predictions when presented with new images.

The Specific Role of Depth Estimation in Autonomous Driving

For autonomous vehicles, depth estimation entails more than just recognizing distances through images; it enables higher-level autonomous driving functions at a lower cost.

Without depth information, vehicles can 'see' their surroundings but cannot accurately judge object distances. With depth information, the 'thinking' of autonomous driving systems can truly progress from two-dimensional to three-dimensional space. Direct applications of depth estimation include:

Collision Warning: Accurately determining the distance of objects ahead to decide whether to brake or swerve.

Path Planning: Calculating the optimal driving route based on three-dimensional spatial relationships, rather than just pixel paths in images.

Following Distance Control: Estimating the distance to the vehicle ahead to decide whether to accelerate or decelerate.

Dynamic Obstacle Prediction: Combining machine learning to track the movements of other vehicles and pedestrians and predict their future positions.

All these functions rely on precise depth predictions. Without reliable depth information, subsequent path planning and control decisions would lack a spatial foundation.

In autonomous driving systems, depth estimation results do not rely on a single source but are fused with data from other sensors such as LiDAR and millimeter-wave radar (i.e., 'sensor fusion').

This approach fully exploits the rich information contained in visual data while compensating for the limitations of single sensors.

Final Thoughts

Viewing depth estimation as merely a module in an autonomous driving system underestimates its significance. It is not a simple image transformation but a bridge that converts two-dimensional vision into three-dimensional spatial cognition.

It enables machines not only to see the world but also to comprehend its structure and spatial relationships. Without accurate depth estimation, autonomous vehicles lack the most fundamental sense of space.

-- END --