Text/VR Gyroscope Wan Li

The AR hardware market has seen optical solutions evolve through years of iterative development, with "Micro-OLED+Birdbath" and "Micro-LED/LCoS+Optical Waveguide" emerging as mainstream choices. However, amidst this landscape, startups are also exploring niche solutions to push the boundaries.

One such innovator is the Swiss startup CREAL, which asserts that traditional AR optics possess "fatal flaws" due to their pseudo-3D nature and inability to address issues like zoom. The company aims to introduce light field technology into AR glasses, offering users a more natural and immersive visual experience.

Image source: CREAL

What is light field technology, and how does it address current challenges?

Founded in 2017 and headquartered in Switzerland, CREAL has been focused on developing light field display technology tailored for VR, AR, and other devices since its inception. Tomas Sluka, the company's co-founder and CEO, brings a strong background in optics and physics from his research tenure at CERN and EPFL. Additionally, the CREAL team boasts members with backgrounds from Intel and Magic Leap, contributing to the company's robust technical capabilities. According to LinkedIn data, CREAL employs between 11-50 people.

Light field technology, at the core of CREAL's efforts, aims to reconstruct the way light propagates in the real world. CREAL explains that "light in the real world can be described as a continuous field of rays reflected, refracted, or emitted by physical objects and propagating through free space. Each point in real space transmits an infinite number of light rays in infinite directions. A digital light field simulates this phenomenon." Essentially, it can be understood as a true 3D representation.

Currently, light field cameras are available on the market. Unlike traditional cameras that capture only the intensity and color of light, light field cameras also record the direction of light propagation. Notably, these cameras can record depth of field, enabling operations like post-capture focusing.

Light field camera launched by EYEMORE Tech, Image source: Internet

Beyond cameras, light field technology can be applied to large screens, AR, VR, and other display terminals. Broadly, light field screens on the market can be classified into two types:

1. Passive light field displays: These include solutions like lenticular lens arrays, gratings, and custom microlenses, exemplified by Looking Glass. These devices preset multiple angles for image transmission, enabling naked-eye 3D display effects. However, their light emission angles are not adjustable, and light efficiency loss is significant.

Image source: Looking Glass

2. Active light field displays: Utilizing components like spatial light modulators (SLM) and MEMS micromirror arrays, these displays can finely adjust the light propagation direction of pixel points, achieving a natural viewing experience. CREAL employs this scheme, utilizing FLCOS SLM + HOE thin-film optical modules in its smart glasses.

Image source: CREAL

Why Introduce Light Fields?

For some, light field technology solves the naked-eye 3D viewing problem, making it valuable in education, training, exhibitions, and displays. While VR/AR already offers 3D viewing through binocular parallax, the urgency of light field technology for VR/AR may not seem immediate from this perspective.

However, a common issue with most current VR/AR products is their limitation to a single fixed focal plane display, often mismatching virtual images with real-world scenarios. (The typical virtual image distance for AR glasses is around 1.5-3 meters.)

For instance, anchoring a virtual image on a 2-meter focal plane close to your hands would result in the virtual image being out of focus when looking at your hands under traditional schemes. This requires the eyes to constantly adjust, providing a poor experience and potentially causing visual fatigue, dizziness, nausea, and other discomforts with prolonged use.

In contrast, CREAL's light field solution allows seamless switching between near and far perspectives, ensuring excellent display effects. The company claims on its official website that this solution offers natural vision, providing a "completely correct and healthy visual experience."

Image source: CREAL

At this year's AWE, CREAL exhibited, and VR Gyroscope experienced the product prototype on-site. Tomas demonstrated using a zoomable camera lens to simulate the human eye, cooperating with gesture recognition to grab virtual objects. Under ordinary display schemes, switching to multiple focal planes and accurately grabbing virtual objects is impossible. However, CREAL's light field scheme allows for such switching, and the distance of the grabbed virtual objects can be arbitrarily adjusted.

Image source: CREAL

For mixed reality scenarios, there are two solutions on the market: VST and OST, corresponding to MR and AR, respectively. From a technical standpoint, VST captures and synthesizes images using a camera, while OST presents virtual images as "stickers" in the real world. Therefore, the OST solution is more prone to vergence-accommodation conflict (VAC) issues.

Why is Light Field the Preferred Route?

In addition to light fields, other technical solutions aim to improve/solve the single focal plane issue in VR/AR, such as multi-focal planes and dynamic zoom. In its whitepaper "Natural Vision in AR," CREAL compares these routes and argues that the light field solution might be one of the best choices for commercialization.

Multi-focal plane display: Introduces multiple depth planes into the device. For instance, Magic Leap Generation 1 has two focal planes, activating a more fitting virtual screen based on eye-tracking data.

LightSpace has showcased an AR glasses prototype with four focal planes, using time-multiplexed display based on a liquid-crystal diffuser stack and a high-refresh-rate, high-speed projector. However, this solution's optical system is relatively complex.

Image source: Internet

Zoom display: Utilizes zoom optical elements and eye-tracking to match the viewing image in real-time. Meta has previously showcased a headset prototype of this solution (codenamed Butterscotch Varifocal). CREAL notes that this solution requires digital simulation of optical blur and ocular parallax, but eye-tracking may have limitations in accuracy, response time, and reliability. Its medium-to-long-term technological advantages remain uncertain.

Computer-generated holography (CGH): Uses a phase-type spatial light modulator to reconstruct a discrete approximation of the target light wave. Theoretically, this is the most ideal solution. However, it currently faces challenges like unstable image quality, high sensitivity to temperature and voltage, low refresh rate, and high computational complexity, keeping it in the laboratory stage.

Through comprehensive comparison, CREAL believes its light field technology could be one of the more ideal solutions to address visual challenges like VAC in AR devices.

Notably, CREAL employs a sequential light field, synthesizing multi-viewpoint images through sequential timing to form a light field with parallax information at the human eye. Some materials indicate that compared to spatial light field displays, sequential light field displays offer higher angular resolution (more viewpoints) without sacrificing the spatial resolution of a single image.

In terms of optical principles, the CREAL system uses a 2D needle light array as the light source. The light is processed by the light modulator and projected onto the retina through a holographic thin film. Each projected ray carries a low-resolution image, and the modulator ensures each viewpoint in the eye receives a specific image perspective. CREAL claims its solution has 36 viewpoints, with an image projection frame rate reaching an astonishing 6500Hz. (Note: Parameters may change with prototype iterations)

Finally, the retina combines low-color-resolution images from multiple viewpoints into a complete image, and the brain "fills in the blanks," enhancing image resolution further.

VR Gyroscope learned from industry experts that CREAL's solution is a retinal display, differing from traditional displays as it's a scanning display, akin to a projection mode. Being scanning-based, it offers a deeper depth of field and zoom capabilities, or its focal length has a relatively large range. However, this solution faces challenges in data processing for light field displays, display quality improvement, optical design, cost, and energy consumption.

CREAL's Commercialization Journey on the Verge of Landing

In 2022, Tomas Sluka mentioned in an interview that CREAL's core goal is not to become a hardware manufacturer but rather a supplier of light field technology solutions. Since its inception, the company has focused on overcoming issues like module size, light efficiency, computational load, and cost, ultimately aiming to introduce light field technology into the consumer market.

Further Miniaturization

Regarding module size, refer to the company's product roadmap. CREAL's earliest prototype was a small box, unsuitable for head-mounted devices. By 2021, the company showcased two wearable device prototypes. The headset adopts a dual-screen design similar to Varjo, with a 1000×1000px resolution light field display in the central 30° field of view and a conventional 1600×1440 display in the peripheral area. The AR glasses resemble HoloLens, supporting a 60° FoV and 1000×1000px resolution display.

Image source: CREAL

In 2024, CREAL released a custom light field modulator (FLCOS) to slim down the module and further miniaturize AR glasses. The latest product prototype is already close to an AR glasses form factor, though the light engine placement in the temple makes it slightly bulky. According to CREAL's latest product roadmap, the company plans to further reduce the light engine size and improve the Eyebox by 2026.

It's worth mentioning that CREAL's solution uses HOE thin films, offering additional benefits: it can be attached to traditional nearsighted glasses, and its transmittance and rainbow patterns are superior to waveguide solutions, providing a natural appearance on the glasses' front.

Image source: CREAL

The image below shows the AR solution specification table on CREAL's website. It reveals that light fields still face display issues like low contrast and weak color performance. Additionally, CREAL has mentioned that when the virtual image is less than 20 centimeters from the eyes, clarity issues may arise, and the modulator's light efficiency is weaker than LBS and Micro-LED, requiring further improvement.

CREAL AR product technical specification table, Image source: CREAL

Application Ecosystem Development

Light field displays differ from conventional displays in their rendering strategy, requiring processing of complex light field information from multiple directions and focal lengths. For common experiences like 3D models and 3D games, light field AR devices need format conversion for viewing.

To address this, CREAL launched a driver compatible with Open XR last year, enabling its light field technology to seamlessly integrate into various XR applications and experiences, enhancing the technology's applicability.

Image source: CREAL

Collaborating with Zeiss for the Optometry Market

According to incomplete statistics, CREAL has received a total of 12 rounds of investment since its inception, raising over $23 million. Investors include Verve Ventures, Zeiss, and others. Notably, Zeiss invested in the company in September 2022 and July 2023, with the latest investment amounting to $8.9 million.

Zeiss's "special preference" for CREAL stems from the promising application of light field technology in optometry. Last year, CREAL revealed, "Our collaboration with Zeiss aims to revolutionize vision diagnosis and treatment by establishing a digital vision care platform. Leveraging CREAL's light field display technology, Zeiss will introduce devices capable of digital eye exams and simulating traditional eyeglasses, contact lenses, and more to the market."

CREAL's authentic 3D display technology allows for the digital replication of any physical or theoretical lens, enabling tailored testing procedures across all age groups and customer segments. It also supports automated content projection, thereby reducing the overall clinical resource demand. Based on CREAL's announcement, it is anticipated that related products may hit the market as early as next year.

Image source: CREAL

Conclusion

From an experiential standpoint, light field technology offers a more natural viewing experience and addresses issues like visual accommodation conflict (VAC), making it uniquely appealing for AR glasses.

However, light field technology is not without its challenges, including high performance overhead, limited resolution, and elevated module costs. Whether VAC is indeed the core pain point of current AR glasses remains to be seen. For instance, Magic Leap has noted that designing a sensible virtual image focal length (1.5-2 meters) and simulating certain visual cues can alleviate VAC to some extent.

While light field technology exhibits technical rationale and superiority, whether it stands as the optimal solution for the future remains subject to further market validation.