abstract visual of virtual reality VR video streaming

There are many complexities involved with presenting spheres on planes, and it’s a key challenge for delivering content that feels immersive. When you put on VR goggles and watch a 360-degree video, you feel that you are in the center of a sphere. Actually, you are inside of a geometric box that only appears to be a sphere.

 

 

Gif from Stanley Kubrick’s 2001: A Space Odyssey

 

At Pixvana, we call this shape a “viewbox”. Our XR Lab is testing a variety of viewbox projection layouts to understand what method best optimizes the quality and speed of VR video streaming. The goal is to create a video stream that maximizes resolution while minimizing bandwidth.

Icon images of field of view adaptive streaming and a photo of a 360 video image of three models in a gardenVisualization of how a projection looks versus how it is perceived in a headset for frustum projection versus a perceived sphereVisualization of Projection vs Perception of an image of a drawn gorilla on a skateboardImages of the 30 VR video viewpoints in a video

 

The Pixvana SPIN Technical Preview (download on Steam for Oculus Rift or HTC Vive) has a special capability where you can experience the effect of projecting a sphere onto a flat surface within a VR video stream. Our field-of-view adaptive streaming (FOVAS) technology optimizes pixel resolution based on where you are looking. In the Technical Preview, you will see the projection of a frustum when you hit pause and expose the viewbox. We have discovered that this shape reduces bandwidth requirements to under 4mbits while delivering ultra high resolution.

 

Frame-by-Frame Tour of the Pixvana SPIN Technical Preview

 

With the headset on, press pause and the still frame will show “foveated pixel density”. The highest resolution will exist at the “front” of the video stream. In this case, the fashion models are at full resolution while the photographer toward the “side” appears at lower resolution:

 

A camera guy taking a picture of two models in a bamboo garden

 

Directly behind the models is the photographer. He appears blurry because he is viewed at the out-of-field area before the switching occurs. However, when you turn your head to look at him then FOVAS will “switch” and quickly increase the resolution. The latency is almost imperceivable at 100 – 200 milliseconds. Here is the photographer when he is viewed at the out-of-field area before the switching occurs:

 

Image of a camera guy in a bamboo garden

 

After the switching, the photographer comes into the current optimized field-of-view, which delivers full resolution from the source file (in this case a 10k master resolution film):

 

Image of a camera guy in a bamboo garden

 

This is what the actual video stream would look like:

 

Image of an actual video stream and how it looks. Image is of camera guy and two models in a bamboo garden.

 

Here is the full 10k frame as a latlong – a file that is way too large for a headset to process or to stream!

 

Latlong image of an actual video stream and how it looks. Image is of camera guy and two models in a bamboo garden.

 

Filed Under: