The Optimal VR headset for VirtualLink
In a previous article on Foveated Rendering, I created a table of resolutions that are possible within the bandwidth constraints of the VirtualLink standard, which is backed by Nvidia, AMD, Valve, Microsoft, Oculus and HTC. The available bandwidth is estimated to be equivalent to DisplayPort 1.4
DSC Compression
Note that DisplayPort 1.4 does also support DSC compression, but for the purposes of this article I am going to ignore it. Basicaly it is fast and visually lossless , and probably superior to chroma subsampling, but best left explored when there are screens that support it.
Our constraint is a threshold of 25.92 Gbit/s
Based on the DisplayPort 1.4 Maximum Total Data Rate, we must send no more than 25.92 Gigabits per second to the display. It is possible that the VirtualLink specification will evolve to exceed this, but if so I will address that in a future article.
With this as a hard limit, I have explored a combination of resolutions, color depths and refresh rates that all fall within this threshold.
Looking at the table I find B-1, B-2 and H-1 the most interesting, because they could all be supported configurations from within by the same headset. There are plenty of precedents for this, the most common being a 4K UHD display being driven by a 1080p source. Also known as True HD, most of the content on Netflix and Amazon is in 1080p.
Why does 1080p content look better at 4K UHD?
A 4K UKD display is 3,840 pixels wide and 2,160 pixels high, whereas 1080p displays are exactly half this amount, they are 1920 pixels wide and by 1080 pixels high, hence the name '1080p' The process in which the 1080p content is stretched to become twice as wide and twice as tall, is called upscaling. And it turns out that when you upscale by a factor of two, the results will always look visually smooth and pleasant.
This is because pixels (a bad abbreviation for Picture element) are a bit like mosaic tiles, arranged in a grid, if you look at them closely enough. The grid structure imposes a limit in that all pixels have the same width and height in the display. But if we are doubling the resolution of our image when we upscale, then the results will be optimal, because we are actually cutting each pixel exactly into four equal quarter size pixels, and the sampling algorithm for this will be optimal.
Lets try this on an image containing text:
The text in the left column in the image is equivalent to 1080p content being shown on a 1080p display. Whereas the text on the right column is equivalent to the same 1080p content being upscaled by a 4K UHD display. While it is not as distinct as native 4K content, it is much smoother and arguably yields a superior image quality.
The H-1 display, at a native resolution of 4,096 x 4,096, can do the same for frame buffers that are rendered at 2,048 by 2,048. It is just bilinear upscaling. So if we chose this display we can support the following resolutions:
- B-1 (2,048 x 2,048) @ 24 bit (R8G8B8) color @ 120 Hz, upscaled to 4,096 x 4,096
- B-2 (2,048 x 2,048) @ 32bit (R11G11B10) color @ 90 Hz, upscaled to 4,096 x 4,096
- H-1 (Native 4,096 x 4,096) @ 8 bit color (256 colors) @ 90 Hz
We basically get x2 upscaling for free when using configurations B-1 and B-2, and an extremely high resolution mode in H-1 itself that is limited to 256 colors which may still be useful for text based applications. B-2 configuration gives us support for HDR color although it is limited to 90 Hz, but this is more than enough for films, and in the case of games its up to the developer to choose this or 120 Hz with B-1.
Overall this is a lot of versatility, three configurations within one headset.
Which Field of View would work best? This is a bit like asking "How thickly do you want your jam spread?" A higher FOV will increase the peripheral vision which will increase immersion. But conversely, this will result in fewer pixels per degree. The Field of View will be shared between B-1, B-2 and H-1, so I'd recommend 137°. This is much larger than today's mainstream headsets which are closer to 100°, and results in 15 native (30 upscaled) pixels per degree, which is equivalent to what the Vive Pro and Oculus Quest are currently offering. But these are smoother x2 upscaled pixels, meaning that four pixels will ocupy the same area as 1, so there will be a substantial jump in quality, as we saw in the image of text upscaling, earlier.
When the headset is being used in H-1 configuration, there are actually 30 native pixels per degree
The most esoteric mode is undoubtedly H-1, even with this extreme resolution would 256 colors be enough to do anything useful with, other than textual applications? In Augmented Reality we could delegate 128 of the colours to be used as a grey-scale/sepia representation of the world that the stereo cameras are seeing, and use the other 128 as 'GUI interface colors'. This should be enough for icons, message boxes, widgets etc. Computers once ran in 256 color mode, so it is certainly achievable. And just think - your AR experience would be unparalleled, at 4096 x 4096 per eye!