In the realm of autonomous driving, the pure vision approach has steadily garnered acceptance and endorsement from certain practitioners and researchers. Through the implementation of binocular and even trinocular camera systems, along with the utilization of parallax calculations, structural constraints, and algorithm modeling, cameras have attained a certain level of depth perception capability. This advancement has progressively broadened their range of applications within autonomous driving perception systems.

Consequently, numerous individuals have grown accustomed to likening pure vision autonomous driving to “driving with the human eye.” However, from a biological vision standpoint, the human eye is inherently constrained by physiological factors and is prone to vision impairments such as nearsightedness. Thus, the question arises: will a camera-based pure vision autonomous driving system also exhibit a comparable form of “nearsightedness”?

What Exactly Is “Pure Vision Autonomous Driving”?

Pure vision autonomous driving, as the term suggests, refers to a vehicle that primarily relies on cameras to “perceive the world.” These cameras, functioning akin to a machine’s eyes, transmit information regarding the road, lanes, pedestrians, obstacles, and so forth, to the autonomous driving system. The system then processes this information to make judgments, plan routes, and control the vehicle’s movements. Compared to sensor fusion solutions, pure vision has garnered significant support from manufacturers due to its lower cost and closer resemblance to how the human eye perceives the road.

From a fundamental perspective, the operating principle of pure vision autonomous driving appears to closely mirror that of the human eye in perceiving the world. At this juncture, one might naturally wonder: given the similarities between machine vision and human vision, will machine vision also suffer from nearsightedness, akin to the human eye? Or, will it perform like a nearsighted eye in specific scenarios, being unable to see distant objects clearly or discern fine details?

To address this question, it is essential to first understand the differences between the structure of the human eye and machine vision.

There exist notable disparities, and even a greater degree of complexity, between how the human eye functions and how a camera operates. The human eye comprises intricate structures such as the lens and retina. It adjusts its focal length through muscular regulation to form clear images of objects at varying distances. Improper focal length adjustment can lead to vision problems like nearsightedness and farsightedness. The eye transmits two-dimensional light information to the brain, which then reconstructs and interprets this information, ultimately forming the world as we perceive and understand it.

In contrast, the camera mounted on a vehicle is more akin to a fixed-focal-length camera. It is installed facing forward and employs a lens and sensor to convert optical images into digital signals. These signals are then transmitted to the autonomous driving system, where they are processed by algorithms to comprehend the surrounding environment. Unlike the human eye, the camera lacks a natural mechanism for “adjusting the focal length” and does not integrate information based on experience, attention, or other senses. It simply “captures” the image and processes these pixels through algorithms.

Is Machine Vision Truly Comparable to the Human Eye?

Machine vision and the human eye are, in fact, not identical. The human visual system consists of two primary components: the eye and the brain. The eye can adjust its focus and adapt flexibly to complex lighting conditions. It also utilizes experience and common sense for reasoning. On the other hand, the “eye” of machine vision is merely a simple image collector. Its depth perception, object recognition, and distance estimation capabilities all rely on algorithms. A single camera cannot directly provide depth information. Consequently, many pure vision autonomous driving systems must estimate distances using algorithms or employ methods such as multiple cameras and stereo vision for indirect supplementation.

If machine vision fails to perceive clearly, it is not due to the “eye being nearsighted” but rather due to the physical limitations of the camera itself and issues with the algorithm’s judgment capabilities. For instance, in situations with extremely weak or intense light, the images captured by the camera may contain noise or be overexposed, affecting the subsequent perception algorithm’s ability to recognize and locate objects. Without auxiliary information such as depth sensors or high-precision maps, the algorithm’s performance in certain complex scenarios may resemble that of a nearsighted human eye, being unable to discern objects clearly. This sensation is somewhat akin to the difficulty in seeing fine details when the human eye is visually fatigued, but in essence, it is not physiological nearsightedness but a technical limitation.

The human eye can swiftly judge the distance and speed of objects, detect changes in light and shadow, and even make inferences about the situation ahead based on experience in challenging environments. In contrast, a pure vision system must rely on algorithms to estimate three-dimensional information from two-dimensional images, a process that inherently involves errors and uncertainties. This uncertainty may manifest as blurriness, difficulty in making judgments, or even misjudgments in specific scenarios. From the user’s perspective, this can indeed seem somewhat similar to the experience of being nearsighted and unable to see distant objects clearly.

Under What Circumstances Will Pure Vision Exhibit “Nearsightedness”?

In situations such as intense direct sunlight, backlighting, weak nighttime lighting, or hazy weather, the image quality perceived by the camera in a pure vision autonomous driving system will decline sharply. With poor image quality, the subsequent algorithm’s judgment will also deteriorate, potentially failing to recognize distant obstacles or incorrectly estimating distances. This scenario is somewhat analogous to the experience of a nearsighted person seeing blurry images from afar without wearing glasses.

Similarly, without high-precision maps or sensors such as auxiliary radar or lidar, the pure vision system’s ability to navigate complex streets and rapidly changing traffic conditions will also diminish. In these extreme, long-tail scenarios, relying solely on cameras may not allow for stable judgment of the situation. This is not inherently nearsightedness but rather a lack of reliable depth perception and supplementary information.

Another aspect to consider is the algorithm’s learning and generalization capabilities. Deep learning models are trained on vast amounts of data and can handle common scenarios effectively. However, their judgments may be unstable in rare situations or when the data does not cover certain cases. Just as humans can make judgments with the aid of experience and other senses (such as hearing and spatial memory) when suddenly faced with rain, fog, or sudden changes in light in a tunnel, a pure vision system can only base its judgments on image data, increasing the risk of misjudgment.

Can the “Limitations” of Machine Vision Be Overcome?

Given the numerous challenges associated with pure vision autonomous driving, are there viable solutions? In theory, these limitations can be gradually mitigated through technological advancements, but it is actually very challenging to replicate the human eye’s capabilities exactly.

Nowadays, many autonomous driving solutions do not rely solely on pure vision but integrate perception hardware such as lidar and millimeter-wave radar with cameras. In this way, when visual perception is weak, millimeter-wave radar and lidar can supplement distance information and environmental depth perception. This fusion solution offers greater stability than the pure vision approach.

In terms of algorithms, technologies such as deep learning, three-dimensional reconstruction, and visual depth estimation are constantly evolving. Many pure vision systems can now enhance their understanding of complex scenarios through software upgrades. For example, visual depth estimation algorithms can infer distance information from monocular images or obtain more accurate depth by using multiple cameras to form stereo vision.

There are also innovative methods that attempt to utilize optical information from different wavebands to supplement the camera’s perception capabilities and improve the stability of visual perception in weak light or complex lighting conditions. A typical approach is to fuse the data of visible light and near-infrared (NIR) spectra. In this manner, the system can not only obtain the image captured by the camera but also leverage the imaging advantages of near-infrared light in low-light or backlighting situations to achieve a more comprehensive perception of objects and structures in the scene.

In conclusion, the perception capabilities of pure vision autonomous driving will undoubtedly continue to improve. In the future, through the development of more robust algorithms, higher-performance cameras, and more intelligent data fusion methods, it may be possible to achieve perception effects that are as good as or even superior to those of humans in most road conditions.

The Final Word

Returning to the question that preoccupies many: will pure vision autonomous driving suffer from nearsightedness akin to the human eye? The answer is that it will not exhibit physiological “nearsightedness” because the machine’s camera does not possess a focus-adjustable structure like the eyeball. Its perception limitations are not characterized by visual blurriness like that of a nearsighted eye but rather by difficulties in processing complex images and depth information at the technical level.

However, in certain lighting, weather, or extreme scenarios, its perception results may appear to be “nearsighted.” Yet, with the continuous progress of algorithms, hardware, and system fusion, these issues will gradually be ameliorated. Nevertheless, to achieve the same level of flexibility and comprehensiveness as human vision, there are still numerous formidable challenges that need to be overcome at present.

-- END --