At GTC 2026, Wu Xinzhou delivered an in-depth presentation on NVIDIA's L4 reference hardware and software suite for autonomous driving, encompassing the OEM ecosystem and mass production strategies. NVIDIA posits that addressing the challenges of embodied AI and autonomous driving essentially boils down to a "three-computer" paradigm: training computers in the cloud, simulation computers, and inference computers deployed within vehicles. Focusing on the pivotal "vehicle inference computer," NVIDIA introduced its latest-generation L4 unified reference architecture for vehicles, centered around Drive Hyperion.

1. Core Computing Platform: AGX Thor In-Vehicle AI Supercomputer

The AGX Thor serves as the flagship in-vehicle system-on-chip (SOC) within this architecture, meticulously designed to cater to the high computational demands of in-vehicle large models and end-to-end autonomous driving systems.

Underlying Architecture: Built upon the Blackwell architecture, the same foundational framework used for training large-scale models.

Computational Precision Support: Capable of handling floating-point numbers ranging from 32-bit to 16-bit, with a particular emphasis on 4-bit floating-point numbers (FP4).

Performance Enhancement: Given that in-vehicle devices are typically constrained by memory bandwidth, the adoption of FP4 effectively overcomes these physical limitations, substantially boosting the available in-vehicle computational capacity. With FP4, the AGX Thor achieves a remarkable 20-fold performance improvement over its predecessor, the Orin platform.

Furthermore, NVIDIA's Thor X chip, with a dual-chip configuration offering approximately 2000 Tera Operations Per Second (TOPS), stands as the mainstream computational platform for NVIDIA's L4 solutions.

2. Standardized Sensor Suite (Reference Design)

To facilitate data sharing across diverse OEMs and platforms, Hyperion provides a standardized sensor hardware architecture, available in two configurations to cater to varying requirements:

High-End Model (L3/L4 Level):

Target Scenarios: Designed to support advanced autonomous driving capabilities, covering L3 and L4 levels on both highways and urban roads.

Hardware Configuration: Equipped with 14 high-definition cameras, up to 4 internal cameras, 9 radars, and 1 LiDAR.

System Characteristics: Features a comprehensive sensor redundancy design to ensure safe operation in the event of single-point failures.

Basic Model (L2+ Level):

Target Scenarios: Prioritizes cost-effectiveness at the Bill of Materials (BOM) level, offering end-to-end functionality from parking assistance to hands-free driving on highways and urban roads.

Hardware Configuration: Comprises 10 cameras and 3 radars (excluding LiDAR).

System Characteristics: A collaborative solution developed in conjunction with customers within the Drive AV system.

3. System Architecture and ECU Deployment

In L4 vehicle configurations, NVIDIA employs a dual-ECU collaborative architecture, enabling the delivery of both L2+ and L4 capabilities to partners from a unified codebase:

Primary ECU: Runs the existing L2+ software stack. Under the L4 architecture, it receives additional sensor data from the satellite ECU, enhancing its perception and processing capabilities.

Satellite ECU (Safety Redundancy): Operates a safety-redundant software stack, primarily based on classical algorithms. In the event of a single-point failure in the primary system, the satellite ECU assumes control and executes a Minimum Risk Maneuver (MRM).

4. Software and Safety Foundation: The Haloes Layer

Built upon the Hyperion hardware, the unified software safety foundation for L4 is Haloes, which delineates a multi-layered software architecture:

Operating System Layer (Haloes OS): Evolved from the traditional Drive OS, it has achieved the highest level of ISO certification (ISO Delta). Supports both Linux and QNX operating systems. Incorporates libraries such as CUDA, TensorRT, and LLM SDK, empowering developers to deploy large language models (LLMs) and generative AI models for autonomous driving in the most efficient manner.

Middleware Layer: Standardizes the "sensor and vehicle abstraction layer," a task that previously necessitated the involvement of hundreds of engineers from OEMs. By integrating this layer into the Haloes OS, NVIDIA significantly reduces the engineering burden on OEMs when deploying the software stack.

System Safety Guardrails: Integrates traditional software stacks as safety guardrails, including active safety software compliant with 5-star ENCAP and NCAP standards. This ensures that developers can directly leverage the platform's underlying safety mechanisms when deploying custom end-to-end models on Haloes.

5. Ecosystem Collaboration and Commercial Roadmap

Strategic Partners:

OEM Ecosystem: Recently expanded the Hyperion ecosystem to include four of the world's top 10 OEMs: BYD, Geely, Nissan, and Hyundai.

Fleet and Data Closed-Loop: Collaborated with Uber to establish a fleet based on the current-generation Hyperion 10 sensor suite for data collection. Both parties aim to deploy L4 technology in 28 cities worldwide by 2028.

High-Level Roadmap:

This Year (2026): Expand L2+ applications to achieve end-to-end autonomous driving capabilities.

Next Year (2027): Commence initial deployment of L4 fleets.

2028: Achieve large-scale deployment of L4 technology in passenger vehicles, in collaboration with partners.

NVIDIA's L4 hardware reference design and algorithms can be likened to the Android suite for L4 autonomous driving, poised for adoption by various companies to collectively advance the realization of autonomous driving. Enthusiasts interested in autonomous driving products can click on "Who Should Read 'Autonomous Driving Product Manager'? What Value Does It Offer?" to explore how to drive the adoption of autonomous driving from a product perspective.

References and Images

Automotive Special Address: Advancing Level 4 Autonomy, the Path to Scalable, Safe AVs and Robotaxis - NVIDIA's Wu Xinzhou *Reproduction and excerpting strictly prohibited without permission*