Foveated Streaming Is The Genius Tech Behind Valve's Steam Frame...

The Steam Frame has a heap of new features to dig into, but I want to focus on one here: foveated streaming. It sounds like a medical issue but I assure you, it's all about ensuring the Steam Frame's wireless connection is strong, stable, and appears high quality.

The Steam Frame offers a few ways to play games, both locally on its own Arm chip and streamed from a nearby gaming PC. As the name suggests, foveated streaming is used during the latter.

"So, a lot of people might have heard of foveated rendering," Jeremy Selan, an engineer at Valve working on the Steam Frame, says. "That's where you use the direction you're looking at to optimise what the device is doing. So it only has to do a lot of work in that area. We are doing something similar to that dedicated to the streaming process."

Foveated streaming requires the Steam Frame's new eye tracking feature. This uses two cameras embedded just above the wearer's eyes, through which Valve can very quickly monitor where a user is looking at any one time. The clearest patch of your vision, that's thanks to your fovea, hence the name.

Connecting to a PC via its own wireless adapter and a dedicated 6 GHz connection, the Steam Frame has a relatively high level of bandwidth available to it. But it also uses two 2160 x 2160 screens and can run up to 144 Hz (in an experimental mode), which combined with the audio stream, is a lot of data to beam across even a high-capacity wireless connection.

To get around these limitations, Valve is only sending the highest quality image to the area directly in front of where a user is looking.

"We can send a very, very high-fidelity representation of the source data for where you're looking, and then spend far fewer bits on the surrounding area," Selan says.

Keep up to date with the most important stories and the best deals, as picked by the PC Gamer team.

"So, you could imagine that if that foveated area, say, represents 10% of the full field of view, it would actually be a 10x multiplication factor in bandwidth, in latency, in robustness."

Rather than send two views (one for each eye), which is usually what's required for VR, Valve is sending four. Two views are sent in low resolution and represent the entire field of view. The other two are only a fragment of the complete picture, but sent in high resolution.

Source: PC Gamer