Imagine taking a few hundred photos of a room, a stage or a whole street, and getting back a 3D copy you can walk around in a headset, so real that light glints off the windows just as it did on the day. That is what Gaussian splatting does, and in the past few weeks it has jumped from research labs into concerts, broadcast streams, Apple’s software and an $8.2 billion acquisition. Here is what it is, and why XR fans keep hearing about it.
What is Gaussian splatting?
Most 3D graphics are built from triangles: a model is a mesh of tiny flat surfaces with textures painted on. Gaussian splatting skips the mesh. It describes a scene as millions of soft, see-through coloured blobs, called “Gaussians” after the bell-shaped curve that defines how each one fades at the edges. Each splat has a position, a size, a direction, a colour and a transparency. Viewed together from any angle, they blend into a photo-like image.
The technique took off in 2023, when researchers at Inria and the Max Planck Institute showed it could be rendered in real time at high quality from ordinary photos. Because the blobs are drawn directly rather than calculated pixel by pixel, a captured place can be explored smoothly, which is exactly what virtual and mixed reality headsets need.
From photos to walk-in worlds
Headsets are where you can try it today. Meta’s Hyperscape Capture, introduced at Meta Connect 2025, lets Quest 3 and 3S owners scan a room in a few minutes; Meta’s servers then turn the scan into a walk-in replica built with Gaussian splatting and stream it back, as Fast Company and Mixed reported.
Apple has joined in too. Its developer documentation for RealityKit, the 3D framework behind Vision Pro apps, now includes a Gaussian splat resource and component on iOS, iPadOS, macOS and visionOS 27, with a sample app called “Gaussian splats on visionOS” and sessions at WWDC26. In plain terms, any developer can now drop a captured splat scene into an Apple app.
Splats that move: concerts and live sport
The newest step is 4D Gaussian splatting: splats that change over time, so a performance can be replayed in 3D from any viewpoint. On 12 October, K-pop pioneer Lee Soo-man launched two new groups, A2O Soul and A2O Lite, at a showcase built around 4D splatting developed with Samsung Electronics, according to the Korea Herald and Korea JoongAng Daily. Many cameras film the dancers at once, and the system rebuilds them in 3D so new camera angles can be created afterwards; Lee called it possibly the “final form” of the camera.
Streaming is catching up. At the IBC broadcast show in September, Nokia streamed moving splats to Samsung phones and XREAL glasses using MPEG standards and DASH, the same adaptive streaming used for everyday video, as Nokia and Radiance Fields reported. The goal, Nokia says, is to make 3D video as easy to stream as normal video is today.
Big money and a common file format
On 28 September AMD agreed to buy World Labs, the “spatial intelligence” start-up founded by AI pioneer Fei-Fei Li, in an all-stock deal worth about $8.2 billion, according to TechCrunch and The Next Platform. World Labs builds AI models that generate and reconstruct interactive 3D worlds from text, images and video; Li is set to become AMD’s chief scientist once the deal closes, expected by the end of 2026.
For splats to spread, apps need to share files. In February the Khronos Group, the body behind glTF (often called the “JPEG of 3D”), released a candidate extension, KHR_gaussian_splatting, that stores splats inside glTF with contributions from Autodesk, Esri, Niantic, NVIDIA and others, as Khronos and AEC Magazine reported. Viewers that don’t understand splats can still show the data as a point cloud. The extension was still a release candidate in the latest reports.
The catches
- Big files. A detailed scene can hold millions of splats, so compression is an active race; separate compression extensions are being proposed alongside the glTF baseline.
- Frozen light. Apple’s documentation notes that splats keep the lighting of the moment they were captured; scene lighting doesn’t change them.
- Capture takes care. Good results need lots of overlapping photos or many synchronised cameras, which is why live 4D capture still relies on dedicated rigs.
Why it matters for XR
Headsets have always been hungry for content that looks real. Gaussian splatting lets anyone with a phone or a headset turn a real place into something you can stand inside, and the same idea now works for moving people. With Apple and Meta supporting it, broadcasters testing it and a common file format on the way, splats are becoming to 3D what JPEG was to photos: the everyday way to capture a moment.
Read next
- Optogenetic therapy: algae protein helps blind eyes sense light
- Build your own AI gadget: Meta’s Muse kit for ESP32 and Pi
- Nvidia RTX Spark: laptops that run big AI models offline
Sources
- 3D Gaussian Splatting for Real-Time Radiance Field Rendering – Inria / Max Planck Institute, SIGGRAPH 2023
- Meta’s Hyperscape turns real places into VR worlds – Fast Company, September 2025
- Meta’s hyper-realistic Quest environments are no longer exclusive to the US – Mixed
- GaussianSplatResource – Apple Developer Documentation (RealityKit, iOS/visionOS 27)
- Gaussian splats on visionOS (sample code) – Apple Developer
- With two new groups under A2O, Lee unveils technology that lets fans experience performances from virtually any angle – Korea Herald, 12 October 2026
- SM founder introduces alien-themed idols, cutting-edge technology in return to K-pop – Korea JoongAng Daily, 12 October 2026
- Nokia showcased pioneering DASH-based volumetric video streaming at IBC 2026 – Nokia, September 2026
- Nokia streams dynamic Gaussian splats over MPEG DASH at IBC 2026 – Radiance Fields, September 2026
- AMD to acquire World Labs – TechCrunch, 28 September 2026
- AMD buys World Labs for $8.2 billion to get AI model cred – The Next Platform, 29 September 2026
- Khronos announces glTF Gaussian Splatting extension – Khronos Group, 3 February 2026
- Khronos announces glTF Gaussian Splatting extension – AEC Magazine, February 2026
