You want to create a game engine that is optimized for virtual reality. What should you consider first?

Being the Founder & Lead developer of Cognac Metaverse, The first most important step is to decide which kind of game you're building, is it for smartphones, PCs or consoles? Which VR options will be suitable for your game, do you need 360 FOV or 180 or 90 depends on the game look, feel and type. Which type of functionalities your game offers, is it for simulation only or does it offer actual interactive gameplay? Will the player be able to grasp all the controls of game easily as number of buttons on most VRs are limited. Do you have the needed time and resources to complete the project? These questions will help you a lot in defining your projects capabilities and focus only on the most important parts of a VR project.

Choosing your rendering and graphics API is about selecting the technology that will allow you to harness the full potential of VR hardware, ensuring that your engine can deliver immersive, high-fidelity visuals with low latency. In game development, I look to prioritize APIs that provide robust support for VR's unique demands, such as stereoscopic rendering and advanced shader capabilities. This choice sets the stage for everything from performance optimization to feature set development, influencing how well the engine can support complex VR experiences. It's a decision that impacts not just the technical architecture but also the ultimate experience of the end-user, making it kery in the success of any VR-focused game engine.

Do NOT do this - if you're building your VR game in DX/Vulkan/OpenGL/Metal etc you are guaranteed to fail; you are reinventing a wheel that much larger teams have already done, for free, and unless you have infinite budget you'll never release a game that's suitable for today's market. The only choices in VR engine that matter are: 1. Hardware abstraction layer (OpenVR, XRToolkit, VRToolkit, etc) or hardware-exclusive (Apple, Oculus/Meta, or 'everyone'). Only Apple and Oculus have large enough installed bases to realistically support them solo - all the others you would go for a cross-platform design. 2. Game-engine: Unity, Unreal, or anything else. Unity and Unreal have had 1st-class support for VR for many years. Other engines: less so

All the advice here is incorrect. Instead: 1. Read the hardware manufacturers' own guides to performance on their devices (they publish detailed free papers and studies and sample code). This varies significantly from company to company. 2. Use your game-engine's built-in profiling to achieve a high frame rate (ideally 120 FPS, but 90 FPS is fine, and 60 FPS is possible but dangerous - will make some players sick) 3. Never drop frame rate: it is better for players if you are "60 FPS 100% of the time" than if you are "Sometimes 120 FPS, but varying up and down by 30 FPS" - the human eye needs predictability. 4. Remember that most VR halves framerate, rendering separately to two eyes. So 120 FPS in your engine may be 60 in reality

Consistent FPS and app internal time for each frame should be considered too. All frames are seen in HMD later than the moment rendering starts. In many cases this doesn’t matter, but sometimes it does: - apps with fast action need to decide how to handle the user seeing a slightly old state of the world - the easy solution is to ignore this delay as HMDs anyway do things to predict HMD pose and correct timing issues Consistent FPS helps HMDs and the app to keep things smooth. Since we are talking about a new VR engine there are things it could do for apps: - predict the poses of fast objects to make them consistent with the “as-seen” time - maintain old states for e.g. hit calculations - predict FPS and skipped frames - keep FPS steady

Having worked in VR for nearly four years now and on a couple of very notable titles performance optimisation was always the key thing to ensuring that the player experience is enjoyable and comfortable. Implement performance improvement practises at the earliest phase and constantly review and improve them. Invest in either finding or making performance profiling tools and then automate that process into a build pipeline and run it at least once a week, review the findings and work on what was found. Look at what the hardware manufacturer's are providing to help, ASW and DRS are great for VR and can do a lot for improvement or mitigating where further work cannot be done.

Read the many published research papers on "motion sickness in VR" - the techniques are often novel and unexpected, and you need to use results that have been tested at scale on real people, not "it works for me" or "it works for my 2 friends I tested with".

For the most part VR apps can be tested like any other game type app. From unit tests to simulating user interactions all are done mostly the same way. Things get more interesting when human perception and physicality of the experience and interaction are considered: - UIs need to have suitable size and distance - Users might not automatically see everything (compared to 2D screen) - 3D objects (including UI) that are too close can feel uncomfortable - Waving hands in the air can be tiring - FPS needs to be steady - Conflicting depth of overlapping sharply rendered surfaces can feel weird - Movement can cause VR sickness for some users Most of the above require testing by real humans and the success is measured subjectively.

VR/XR HMDs do many things to make the UX good. There are also differences in FOV, FPS, etc. compared to screen setups. These should be understood and taken into account when designing engines and apps. Methods used for non-immersive displays remain valid, but there are additional considerations: - HMD time warping needs to be taken into account when rendering at lower FPS - FOV is often much bigger than a typical monitor. This makes e.g. flickering shadows near the FOV edges very annoying - VR/XR fills the FOV and this affects how brightness, color and dynamic range are perceived - HMDs require rotating head (not just eyes) more than display based games/apps - Hand waving in the air could have health effects in the long run - ...