Moments ago, Li Feifei took to social media to announce that her AI venture, World Labs, has officially rolled out the Marble world model. Starting November 12, this innovative tool will be accessible to all.

Two months prior, World Labs offered a sneak peek at Marble, a world model adept at crafting 3D realms from image or text cues. A select group of early testers has already had the chance to create their own 3D worlds using Marble.

Today marks the official public debut of Marble by World Labs, a trailblazing generative multimodal world model.

Simultaneously, Marble's capabilities have undergone a substantial enhancement, encompassing the following:

Multimodal Marble: This version can conjure up 3D worlds from text, images, videos, or basic 3D layouts. Moreover, it empowers users to interactively tweak, expand, and merge these worlds. The resulting 3D worlds can be exported in formats such as Gaussian splats, meshes, or videos.

Marble Labs: This platform serves as a showcase for artists, engineers, and designers to display inventive, real-world workflows and explore fresh avenues in gaming, visual effects, design, robotics, and beyond. Marble Labs also delivers comprehensive case studies, tutorials, and documentation, equipping everyone with the necessary resources to learn, construct, and share their own 3D worlds.

Experts in the field propose that world models should be inherently multimodal, capable of transforming any given input signal into a fully realized three-dimensional world and continuously refining their comprehension of the world as new data surfaces.

Marble empowers users to swiftly generate complete 3D worlds from straightforward image or text cues or to interactively edit 2D and 3D worlds, thereby materializing the imagined realms in their minds.

Marble has the ability to craft entire 3D worlds from a single image or a succinct text prompt, producing worlds of diverse scene types and artistic styles. The image prompt feature also simplifies the integration of Marble with other AI tools.

For the effortless creation of worlds with greater creative control, multi-image prompts offer a straightforward and effective approach. In contrast to text or single-image prompts, where Marble must fill in the gaps for unmentioned aspects of the world, multi-image prompts enable you to dictate how the generated world appears from various perspectives.

Marble also incorporates built-in AI-driven world editing tools. Edits can range from minor and localized, such as removing an object or touching up an area, to extensive, like swapping objects, altering visual styles, or restructuring sizable portions of the world.

World Labs has also introduced Chisel, an AI-driven tool for directly sculpting marble worlds in 3D.

Chisel represents a novel experimental editing mode tailored for advanced users to craft 3D worlds. Users can employ basic 3D shapes to outline the rough structure of a 3D world or import pre-existing 3D assets into the scene.

Subsequently, text prompts can be incorporated to define the visual style of the scene or to incorporate additional elements not included in the rough layout.

Chisel distinguishes between structure and style. The rough 3D scene dictates the world's structure, while the text prompt governs its overall aesthetic. These two elements can be combined at will, introducing a new layer of control to world generation.

Following world generation, Marble facilitates one-step expansion to enlarge the world. Users can select the area of the world to expand, and Marble will generate additional content to fill the chosen area.

Originally fragmented world areas become clear and complete post-expansion. Expansion can also be employed to infuse specific areas of the world with greater detail.

After crafting a world with Marble, users have multiple export options at their disposal.

Gaussian splats represent the highest-precision rendering technique for simulating marble worlds. Users can render Gaussian splats in the browser using Spark, an open-source, cross-platform renderer integrated with World Labs and THREE.js.

Marble worlds can also be exported as two types of triangular meshes: collider meshes and high-quality meshes. Once exported as meshes, these worlds can be seamlessly integrated with numerous industry-standard tools.

Marble can also enhance exported videos, adding intricate details, eliminating imperfections, and injecting dynamic elements into the scene while maintaining pixel-perfect camera control and adhering to the structure of the generated 3D world.

Key personnel from World Labs have stated that Marble represents just one step on the path toward spatial intelligence. Looking ahead, interactivity will emerge as a pivotal opportunity. Future world models will empower humans and agents to engage with generated worlds in entirely novel ways, unlocking a plethora of application scenarios in simulation, robotics, and other domains.

References:

https://www.worldlabs.ai/blog/marble-world-model