Before diving into today's topic, let me clarify: for vision-only autonomous driving, nighttime is indeed an extremely challenging scenario. Today, we will solely discuss the technical feasibility of vision-only autonomous driving in perceiving nighttime scenes.

Many experienced drivers can relate to the strain of driving at night, as our retinas have limited light-capturing capabilities—a limitation also shared by cameras in low-light conditions. However, with explosive advancements in hardware technology and deep learning algorithms, the 'vision' of vision-only systems has undergone a transformation in the dark.

How does hardware capture light at night?

To address nighttime perception, the first hurdle is visibility. While automotive cameras share principles with smartphone cameras, their underlying logic differs significantly. To capture clear images in faint light, autonomous vehicles employ large-format CMOS sensors. These sensors feature larger individual pixels, enabling them to capture more photons simultaneously—akin to using a bigger bucket to collect rainwater; even in sparse rainfall, a larger bucket collects more water. Similarly, sensors with larger pixels ensure better baseline image brightness in nighttime scenes.

Beyond light intake, High Dynamic Range (HDR) technology acts as a safeguard for vision-only systems at night. Nighttime road lighting is extremely complex, with pitch-black alleys and blinding high beams from oncoming vehicles. Ordinary cameras struggle: focusing on shadows overexposes highlights, while focusing on highlights leaves shadows pitch-black. Automotive sensors use multi-exposure techniques to capture images at different brightness levels within milliseconds and fuse them, ensuring the system can still read license plates behind glare and distinguish pedestrians in shadows even under direct strong light.

Moreover, automotive cameras have a wider spectral response range than the human eye. Some sensors detect faint infrared light. In pitch-black wilderness without streetlights, what appears as solid darkness to the naked eye can still yield subtle light reflections captured by cameras. These raw data undergo noise reduction in specialized image signal processors, filtering out clutter to produce digital images with clear outlines and features, even if colors are muted.

How do neural networks 'imagine' scenes in the brain?

With high-quality imagery secured, the core challenge becomes interpretation. During the day, road features are distinct, making AI identification of vehicles or lane markings relatively straightforward. At night, however, many visual cues blur or vanish. This is where deep learning models excel. Modern algorithms no longer rely solely on edge detection; instead, through massive data training, neural networks develop robust feature-extraction capabilities. Even a pair of faint taillights or a shadow swaying in darkness can be swiftly identified as a vehicle or a pedestrian pushing a bicycle by data-trained models.

Current algorithms also incorporate temporal information. Vision-only systems don’t process isolated frames but ingest continuous video streams. If the system detects a pedestrian at an intersection one second ago, even if they step into an unlit shadow the next second, the algorithm retains the target in memory based on motion vectors and historical data. This time-series-based perception compensates for instantaneous visual signal loss in extreme conditions, ensuring coherent environmental understanding.

This understanding extends beyond mere recognition; it includes adaptation to lighting changes. Algorithms learn to filter out road glare, rain puddle reflections, and neon light interference. Through iterative training on large-scale real and simulated nighttime scenes, neural networks extract critical driving information even under extremely low signal-to-noise ratios—much like an experienced driver inferring road potholes and obstacles despite blurred vision.

How is lost depth information recovered?

A common critique of vision-only systems is their lack of LiDAR for nighttime distance judgment. During the day, ample lighting and rich detail allow systems to compute precise depth via stereo vision or monocular ranging algorithms. At night, however, lost detail complicates depth estimation. To address this, vision-only systems increasingly adopt Bird's Eye View (BEV) perspectives.

Under the BEV framework, the system converts 2D images from multiple cameras into 3D spatial information in real time. This process relies not on simple geometric calculations but on technologies like Occupancy Networks, which divide space into tiny grids. The algorithm assesses whether each grid is occupied. Even if cameras can't discern object textures, subtle light variations allow the system to detect volumetric masses in space. This shift from object recognition to spatial perception equips vision-only systems with robust nighttime spatial modeling capabilities.

To further enhance accuracy, these systems integrate the vehicle's motion state. As the vehicle moves, cameras at different positions observe the same area from varying angles. Through multi-dimensional cross-validation, the system reconstructs a 3D map of the surroundings in real time, accurately measuring distances to preceding vehicles, curb heights, overhanging branches, and complex construction barriers.

Final Thoughts

The logic behind vision-only systems' nighttime perception is a synthesis of hardware light-capture capabilities, data-trained interpretive power, and spatial reconstruction. Rather than merely mimicking human eyes, these systems leverage silicon-based computational advantages and massive data memory to construct a perception framework that is more acute and rational than carbon-based organisms in the dark.

-- END --