What is Foveated Rendering
Foveated rendering is based on how our eyes work. We only see a small, central spot in super clear detail. Everything else is sort of blurry. Rather than using power to render everything perfectly, foveated rendering focuses on what you're actually looking at.
By cutting back on resolution in your peripheral vision, apps - especially VR ones - can run smoother and faster. It gets its name from the fovea, that tiny part of your eye that gives you sharp vision. It's a clever trick to make VR look great without killing your hardware.
Why Foveated Rendering Matters for VR
Making big, detailed 3D worlds in real-time is tough. Even high-end computers struggle when every single pixel has to be perfect. Foveated rendering steps in by focusing power on what you're looking at and simplifying the rest.
NVIDIA says that using foveated rendering with Variable Rate Shading (VRS) can drop the pixel shading workload by up to 50% (Source). This lets you have:
Cooler lighting
Sharper textures
Faster frame rates
For you, that means fewer stutters, smoother images, and a better VR feel. This is important, because lag or dropped frames in VR can ruin the experience or make you feel sick, VR foveated rendering helps minimize this. (Source).
How Foveated Rendering Actually Works
Basically, it matches the detail to what you can see. It breaks the screen into zones:
The center (where your fovea is) gets full detail.
Everything else gets less resolution as you move outwards.
If you throw in eye-tracking, it gets seriously smart. VR headsets like the Varjo XR-4, Meta Quest Pro, and PlayStation VR2 watch where your eyes are looking. They then shift the high-res zone in real-time. This is eye-tracked foveated rendering.
Then there's dynamic foveated rendering. These systems try to guess where you'll look next and get those areas ready. This keeps things smooth, especially when you move your head fast.
The 3 Kinds of Foveated Rendering
There are several types of foveated rendering techniques, each with its own advantages. These are:
Static Foveated Rendering: A fixed, high-quality area that doesn't move. Useful for systems without eye-tracking, but not as immersive.
Dynamic Foveated Rendering: Changes the high-res area based on your actions or guesses where you're looking. More responsive than static, without needing eye-tracking.
Eye-Tracked Foveated Rendering: Uses real-time eye-tracking to pinpoint the detailed area. Gives the most natural and efficient experience.
AutoVRse creates VR solutions including eye tracking, and performance solutions. For better images and smooth experiences, connect with the team to understand how VR foveated rendering can support smoother, more efficient training or design simulations.
Benefits of Foveated Rendering
VR foveated rendering offers multiple benefits. Here are a few:
Better Frame Rates: Since fewer pixels are fully rendered, the processing needed is lesser.
Less Power Use: Longer battery life + less heat on wireless headsets.
Improved Visuals on Basic Hardware: More quality in focal areas even on cheaper systems.
Reduced VR Fatigue: Maintains better frame-pacing leading to lesser motion sickness and discomfort.
Applications of Foveated Rendering
Foveated rendering is largely used to augment rendering performance. Here are some ways in which it is currently being used.
Gaming: Games like No Man’s Sky (PS VR2) and Microsoft Flight Simulator (Varjo Aero) make it work well without tanking performance.
Simulation and Training: Flight simulators use it to focus the detail on what matters, which improves the realism and lowers compute demands.
Design and Engineering: Car brands like Ford use foveated rendering to see how comfortable elements are in new car designs (Source).
Healthcare and Research: Science labs use VR setups with foveated rendering to study attention, and action in realistic environments, keeping distractions to a minimum.
Tools That Make It Happen
Here are a few tools that enable VR foveated rendering:
Eye-Tracking Headsets: VR headsets (Varjo XR-4, Meta Quest Pro, and PlayStation VR2) have precise sensors.
Real-Time Rendering Engines: Software platforms, like Unity and Unreal Engine, support it.
GPU Architecture: Graphics cards (NVIDIA RTX and AMD RDNA 2/3) can control pixel detail across the screen.
What's Next for Foveated Rendering
As eye-tracking becomes more common, foveated rendering will likely be a normal feature found in VR apps. AI will be used to predict where you're going to look and render the focal zones in real time. It is also seen in machine learning.
In Augmented reality/ Mixed reality, expect foveated rendering to play a part in visual delivery and keeping weight to a minimum.