LiTo: Surface Light Field Tokenization Unifies Geometry & Appearance in 3D Vision (2026)

LiTo: Surface Light Field Tokenization Unifies Geometry & Appearance in 3D Vision (2026)

Introduced in March 2026 by Apple and leading AI researchers, LiTo: Surface Light Field Tokenization redefines 3D object representation by jointly modeling geometry and view-dependent appearance in a single latent space. Unlike traditional methods, LiTo captures specular highlights, Fresnel reflections, and lighting-dependent color shifts from just one RGB-depth image — unlocking unprecedented photorealism in AI-generated 3D content.

How LiTo Unifies Geometry and Appearance

LiTo treats RGB-depth pairs as sparse samples of a surface light field — a mathematical model of how light interacts with surfaces across all viewing and illumination angles. By encoding these samples into compact latent vectors, the model learns to reconstruct not only 3D shape but also material reflectance properties like metal, glass, and glossy plastics — all within a unified representation. This eliminates the need for separate geometry and texture pipelines, a longstanding limitation in neural rendering.

Advantages Over NeRF and 3D Gaussian Splatting

Traditional neural radiance fields (NeRFs) require 20–50 input images and struggle with occlusions or textureless surfaces. 3D Gaussian Splatting demands high computational overhead for real-time use. LiTo, by contrast, achieves state-of-the-art results from a single RGB-D frame, reducing data acquisition costs by over 90%. According to ICLR 2026 peer review (OpenReview ID: TVP0p4f2Su), LiTo outperforms both methods in geometric fidelity and photorealistic rendering under novel lighting conditions.

Real-World Applications in AR, Robotics, and E-commerce

With pre-computed latent codes enabling real-time rendering, LiTo is ideal for next-gen AR/VR experiences, digital twins, and AI-powered product visualization. Apple, already embedding LiDAR in iPhones and iPads, is poised to integrate LiTo for spatial computing. E-commerce platforms can generate photorealistic 3D product models from single smartphone photos, while robotics teams can rapidly simulate environments using sparse sensor data. The approach works robustly across ShapeNet, CO3D, and real-world iPhone LiDAR captures.

Limitations and Future Roadmap

Current challenges include sensitivity to low-quality depth sensors and reduced accuracy with highly transparent or translucent materials. However, the team demonstrates strong generalization across diverse datasets. An open-source release of code and models is anticipated in Q2 2026, following peer review — a move expected to accelerate adoption across academia and industry.

LiTo: Surface Light Field Tokenization marks a pivotal leap toward true 3D understanding, where geometry and appearance are no longer treated as separate entities but as inseparable components of a unified physical model. This breakthrough sets a new benchmark for single-image 3D generation and paves the way for the next era of neural rendering in 2026.