Metal 2 Brings Interesting VR Rendering Essentials to MacOS

Image credit: Apple

Back in June, a month ago at the WWDC 2017 Conference Apple, announced some of the interesting developments with regards to virtual reality. A major computing hardware overhaul has been introduced to its coming line of iMac Pro PCs which is more than enough to meet the high-end VR graphic and processing requirements.

Apple has now announced to incorporate special rendering features to its Metal 2 Graphics API. Apple says that the update Metal 2 API “provides powerful and specialized support for virtual reality (VR) rendering.” This specially tuned rendering between the GPU and the VR headset will ensure a smoother and richer VR experience while meeting up the demands of low latency and high-end graphics.

Let us have a look at the new rendering features in Metal 2 for VR.

Direct-to-Display

Also known as Direct mode, this rendering feature allows the GPU to direct have an access to the VR headset through bypassing other parts within the pipeline which are only important for traditional monitors.

Image credit: AMD

In absence of Direct-to-Display, the VR headset is seen as a monitor and would just extend the OS view on this headset. This method involves post-processing initiated by the OS for modifying the rendered image which adds up to the complexities of the virtual reality rendering pipeline. This might cause performance hits, latency issues and tampering of images.

With the Direct-to-Display feature, your VR headset is not considered as a monitor. As a result, the existing window layout and display arrangement remain as it is. Due to this, the VR compositor will directly present the rendered image to the VR display without the need for any OS-based post-processing techniques. Take a look at the video below to understand this better.

https://www.youtube.com/watch?v=dHuDTUcJURU

Thus, with Direct-to-Display, VR rendering pipeline is not suffocated with heavy processing tasks thereby resulting in better user experience. In short, this feature ensures that the rendered images line up automatically to the VR display without involving the OS.

Single-Pass Stereo

Under the traditional modus operandi, to show a 3D image on the VR headset, it needs to be rendered as an individual view for each eye. Some efficient rendering techniques like Single-Pass Stereo eliminate the need to render the same image separately for the each eye. Rather, it allows the GPU to render the image, in a single draw call, for both the eyes.

With Apple’s new Viewport Array Feature, Metal 2 makes this a reality by allowing “per-primitive viewport selection in the vertex shader.” Rather than defining two different images for the left and right eye, developers now create a texture with a resolution of both the eyes and perform the rendering across the entire texture in one pass. The renderer is told about which eye each primitive belongs to in order for the image to get rightly rendered at proper eye region within the texture. Moreover, due to the distance gap between the two eyes, the renderer introduces the correct offset to achieve a slightly different perspective for each eye.

Metal 2 API will arrive along with MacOS version ‘High-Sierra’ which is scheduled to arrive later this year and will be supporting all the Macs that are currently compatible with the existing Sierra version.