What is Gaussian Splatting?
Gaussian splatting, more precisely 3D Gaussian splatting (3DGS), is a technique for representing and rendering 3D scenes. Instead of building a traditional polygon mesh, it models a scene as a large collection of 3D Gaussian distributions: small, elliptical blobs positioned in 3D space, each with properties defining colour, opacity, size, and orientation. When rendered together from a given viewpoint, these splats reproduce the appearance of the original scene with high photorealism.
The technique was formalised in a 2023 paper from INRIA and has attracted significant attention in computer graphics and XR development. The key advantage over its predecessor, Neural Radiance Fields (NeRF), is rendering speed. 3DGS can produce real-time frame rates suitable for interactive VR, not just pre-rendered video.
How Gaussian Splatting Works
Gaussian splatting starts with real photographs and uses a GPU training process to reconstruct a scene as millions of positioned 3D blobs, each encoding colour, opacity, and shape.
From Photos to Splats
To create a Gaussian splat scene, you take a set of overlapping photographs of a real environment, from a phone camera, a drone, or a dedicated camera rig. These photographs are fed into a training process, typically running on a GPU over several hours, that determines the position, shape, size, colour, and opacity of each Gaussian in 3D space.
The training optimises these properties so that rendering the model from any viewpoint matches the original photographs as closely as possible. The more photographs, taken from more angles, the better the reconstruction.
Real-Time Rendering
Once the Gaussian model is trained, rendering a novel viewpoint is a GPU rasterisation operation rather than a neural network inference, which is why it's fast. Current 3DGS implementations render complex scenes at 30 to 90 FPS on consumer graphics hardware, making them practical for real-time interactive VR where maintaining 72 to 90 FPS is essential for user comfort.
This is a substantial improvement over NeRF, which typically requires seconds to minutes to render a single novel viewpoint.
Applications in XR
Gaussian splatting has practical applications across industrial VR training, remote site access, and ongoing training environment maintenance.
Facility and Environment Capture
For industrial VR training, Gaussian splatting can turn a real factory floor, offshore platform, or hospital ward into a photorealistic VR environment in days rather than weeks. Compared to traditional 3D modelling from CAD data or even photogrammetry meshes, the visual quality for complex, cluttered real-world environments is often noticeably better, particularly for areas with intricate machinery, weathered surfaces, and materials that are difficult to model manually.
Workers training in a 3DGS-captured environment are visually immersed in something that looks like their actual workplace.
Virtual Site Visits and Remote Inspections
A 3DGS capture of a site can serve as an interactive virtual tour, allowing remote stakeholders to navigate the space before physically visiting. For industries like construction, real estate, energy, and mining, this has practical value for planning, procurement, and safety reviews. Inspectors can walk through a captured facility and identify issues without incurring travel costs.
Training Environment Creation and Maintenance
Training teams can capture real environments once and reuse the 3DGS model indefinitely, updating it periodically as the physical space changes. This is more cost-effective than commissioning 3D modelling work for each training module, and it keeps training environments visually accurate to the actual workplace rather than an idealised generic version.
As facilities change, with new equipment installed or layouts modified, a re-capture session updates the training environment to match.
Current Limitations
Gaussian splatting models can be large in file size, ranging from tens to hundreds of megabytes for complex scenes, which creates streaming and storage challenges for some XR deployments, particularly cloud-streamed or mobile AR applications.
Editing the scene after capture is harder than with traditional 3D models: you can't cleanly move a piece of equipment in a splat capture the way you can in a CAD-derived scene, because the splats represent appearance rather than geometry.
Dynamic objects, including moving machinery and people, also don't work well in standard 3DGS captures, though research into dynamic Gaussian splatting is active. These are engineering challenges being actively worked on rather than fundamental limitations of the technique.
Where It's Going
Gaussian splatting is developing fast. Compression methods are reducing model file sizes, editing tools are emerging for post-capture modification, and the quality gap between 3DGS and professional 3D modelling is narrowing for environment capture use cases.
Integration into mainstream XR development tools like Unity and Unreal Engine is progressing, making 3DGS more accessible to production development teams. For XR content creators working with real-world environments, it's becoming a first-choice approach rather than an experimental one.
