Photorealism is not desirable in VR. Graphics which do not cause discomfort are.

I see many people talk about the graphics of VR and the subject of photorealism. Many believing photorealism is a complete necessity for VR and is holding it back.

What strikes me as weird about this preoccupation with photorealism for VR is that many people believe this is necessary without ever trying it. They haven't spent hours in a hyper-realistic environment in VR, then hours in some other type of aesthetic, to really get a sense of what these environments are like in VR.

This is an issue pervasive to many things in the VR industry right now. So many people just sit in zoom calls, stare at spreadsheets, e-mail, powerpoint slides and drab office meeting rooms imagining what they want VR and "The Metaverse" to be. But they don't put in time in a VR headset to develop an intuition on what is really enjoyable through actually experiencing the technology. Consequentially people just end projecting all of the expectations of screen-based content onto VR.

Photorealism I think is one of these things. It's an expectation created by screen-based content that is assumed to be relevant in VR. It doesn't come from the majority mass of people who spend a lot of time in VR headsets. As what you find reflected in the most popular VR titles right now is that photorealism is not something that trends at the top of VR charts. I do not believe this is due to technical limitations of current hardware, as graphics could be moving more in a photorealistic direction amongst the most popular VR titles. There is still plenty of room with existing hardware to make it more realistic right now, but it's not going that direction. You do not find an arms race pervasive among the majority of VR users to be in the most realistic graphics possible akin to the AAA console crowd. From various observations and personal experiences of mine, I have come to believe this is because many of the graphical techniques used to create photorealism are uncomfortable during prolonged usage of VR. They cause eye strain, headaches and even nausea. Something can look good if you just pop your head in the VR headset and look around for 30 minutes occasionally. But just simply looking good isn't going to lead to large-scale adoption with VR as you need users to keep the headset on for more than 30 minutes and they need to come back regularly. To get a sense of what is comfortable and enjoyable during prolonged usage of VR you need to spend 4 to 8 hours in a VR headset regularly to get a sense of how various graphical techniques affect you.

Here are some of the graphical techniques which I find cause discomfort in VR.

Any level of bloom tends to feel like you're viewing the world through a dirty pair of glasses or fogged up glasses. It is annoying and I find can lead me to have headaches, as my tendency when having fog on my glasses is to squint and strain. This is something that platforms like VRChat are providing a great testbed for. As users who upload worlds can apply different bloom settings to the world then you can see which worlds get popular and which ones don't. Something that has stood out to me is that the most popular worlds either have no bloom, or they have a bloom slider on the wall that you can adjust. Then conversely any world with prominent bloom that can't be turned off just doesn't get popular. I think the reason is due to bloom being annoying to sit in for hours on end. The first thing I do when joining a world with any prominent amount of bloom is look for the slider to turn it down or turn it off entirely. If someone I know does not already do this, after I point it out to them and they realize the subtle annoyance that bloom causes, then they start to do it. When I first started using VR I remember initially liking a small amount of bloom as it can make things feel "warm" but after getting used to spaces with zero bloom, any level annoys me now, it feels like my lenses are dirty or fogged.

HDR light levels adjusting with adaptive photographic tone mapping are not enjoyable to sit in for hours either. What I am referring to is how games with HDR adjust the overall brightness of the entire screen depending on if you're looking at something brighter or darker. The issue with this in VR is, if you look around your real-world room right now at the various fluctuating light levels, from your actual perception it does not appear like your entire vision is brightening or darkening. Changing brightness levels due to our eyes dilating is generally imperceivable to us. When you try to simulate this in VR it creates an effect where the brightness of the world feels like it is turning up and down as you look around. It's annoying. Imagine if you had a lamp in your room with a brightness slider and as you looked around your room the brightness of that lamp adjusted based on where you looked, that is what it feels like in VR and it's funky.

High levels of specularity and reflections are also not enjoyable to sit in for hours. In VR you end up having movement all around you on all the reflective and specular surfaces as you move your head and the reflection angle changes. Even just sitting still ends up with a good amount of movement on these surfaces as your never perfectly still. All the reflective and specular movement on surfaces tend to catch my eye in VR. This creates a tendency for my eyes to keep glancing to different shiny surfaces, darting to my periphery and back. Due to this any surfaces with a noticeable level of reflectance or specularity just annoy me and cause eye strain. Creators of VR worlds could tone down the specularity and reflectance of surfaces, make it more subtle. But something I realized about this is, spaces you sit in for hours on end in real-life tend to be matte and non-reflective. I think this is partly the reason why people paint the walls of their house with matte paint or buy furniture with a matte finish. Sitting in a space for hours on end with lots of reflections and specularity moving all around your peripheral vision can be annoying. Spaces we sit in for hours on end we prefer to be predominately filled with matte surfaces. Once I got used to being in virtual worlds that had absolutely zero specularity and reflectance, I actually came to prefer it. Now when I go into worlds with even subtle levels of specularity, I do notice it and want to turn it off.

Brighter worlds, high frequency detail and high variation of contrast between objects are also qualities which I find cause discomfort in VR. The more detailing you have in a world, the more grain you have on textures, the more contrast you have between objects and the brighter things are, the more you have things which your eye can catch on. So as you rotate and move around your eye will catch on something, release, catch on something else, release over and over. This creates more eye strain as you move. I also suspect it creates a greater sense of movement and is something that contributes to nausea. If you wanted to create a world that would produce the least amount of eye strain and nausea possible as you ran through it, you would not want any textures, all surfaces should be smooth. For lighting you would want something that would produce smooth gradients across surfaces without sharp edges from shadows or light variation. Overall brightness levels you'd want to be a dimly lit room. If there were different objects you would want their color variation to be similar enough that they tended to blend together in your peripheral vision. Of course, this would be a boring world, but if you were to take creating a no-nausea world to the extreme, from my experience that is what I would do. World creators need to balance this overt simplicity with some detail to make it interesting. This is another aspect you can also see as a trend in VRChat. There are beautifully detailed AAA quality worlds in VRChat. Some worlds are direct imports of photo-realistic architectural visualization scenes, but such worlds never trend as the most popular. The consistently most popular worlds tend to be very simple. They have relatively undetailed textures, more uniform lighting, overall are dimly lit and have more homogenous color palettes.

Again, I don't think this is because of performance issues, you can make more realistic and flashy worlds with great performance. In fact, there are some highly detailed, stimulating and realistic worlds that can be popular for specific things in VRChat. Such as a world used to host raves and other musical events, they can be filled lasers, lights, and even distortion effects intended just for visual stimulation. But such worlds are something people spend a constrained amount of time in, an hour or two, just for the musical event, then they leave because it would not be comfortable to be in such worlds for 4+ hours. There is a usage for overly stimulating VR graphics, but it needs to be understood that won't be the baseline, it will be something occasional for when someone wants that level of stimulation, it is not what will get the most use nor be the most universally desirable. Pretty much in the same exact way no one would want to live in a nightclub, they go to one for a few hours and then leave to somewhere much more relaxed.

I encounter many people who claim that VR makes them nauseas, causes eye strain or gives them headaches because they put it on a few times, ran around some world, had an issue and then assume that is just how VR affects them. But I suspect this could be highly dependent on what sort of content they were viewing, and not just due to them running around a world or standing still, but rather how it visually looks. I suspect more people could run around a dimly lit smooth shaded world with a homogenous color palette for hours with less discomfort than just sitting in a bright highly detailed world with all the HDR bells and whistles for the same amount of time.

Notice that of the things I have described as being uncomfortable in VR, the most popular VR titles tend to not have those things. Beat Saber is composed of smooth glowing blocks on a dark background, no detailed textures, smooth gradients, only a few colors in the palette and little going on in your peripheral vision. Rec Room is all smooth shaded low-detail cartoony graphics. The most popular VRChat worlds tend towards simplicity as well.

Even though what I mention is not consciously known or explicitly said by people, I suspect these are aspects which are unconsciously affecting people's experience and choices. In many ways these aspects of discomfort I have highlighted are at odds with "Photorealism". Imagine achieving photorealism without any bloom, HDR, tone mapping, reflectivity, specularity, high resolution textures nor high-poly meshes. You could come up with something "realistic" within those constraints but it's not going to be the hero shots the industry currently puts forward as the ideal of "photorealism". So many of these techniques made sense for the medium of a screen because we were very literally trying to replicate film, which very literally was trying to replicate the physics of camera optics. But with VR we are no longer doing that, we aren't replicating a camera optic, we are trying to replicate reality as it is perceived through a human mind.

This is an odd thing about VR that designers and graphics people will have to contend with as we go further into it. There is no reason to believe that the physics of light and the physics of a camera optic describe a perception of form that is most easily digestible, and desirable, for our minds. Physical reality didn't evolve to suit our visual perception, rather some aspects of our visual perception evolved qualities to deal with physical reality. What looks "real" in physical reality wasn't created to specifically be the most congruent with the way our mind processes vision. With VR we can create things that are more specifically attuned to how our mind processes vision. I think people would choose to get rid of many perceivable qualities of physical reality if they could, it's just that in physical reality you can't, so we learn to deal with it. But if given the choice some would get rid of light glaring in their face, overt variations of light that cause excessive pupil dilation, unnecessary clutter, movement in peripheral vision, atmospheric haze etc. What people are going to be attracted to in VR is not photorealism but rather that which is most comfortable to spend prolonged periods of time in, that which is most compatible with how their mind processes vision. This may be quite different than the aesthetic you find popular among current AAA graphical techniques and Hollywood films.

On this subject I'd like to note that preoccupation with photorealism is also cultural and dependent upon someone's generation. For various reasons older people in America and various western nations tend to have greater preoccupation with photorealism. I suspect it's because of the mass popularity of Hollywood live-action films and such films coming from America. But you go to another culture like Japan and anime is widely accepted to be as relevant of an aesthetic as live-action film, if not more relevant. Anime has also become mainstream with younger generations in every nation. The younger demographic is also the demographic that is going to care the most about VR. It's possible that on this point alone anime aesthetic will entirely usurp the metaverse simply because fans of that style are already attuned to an aesthetic that better translates into VR. Anime aesthetic already dominates in VRChat. This highlights something that I think is missed, that being, photorealism is not a universal absolute of what is desirable in art, it is dependent on culture. The realist movement of art was just one art movement, it was not the end goal purpose of art. After the realist art movement, art then veered the opposite direction in movements like modernism and expressionism. I think VR is going to veer towards a new kind of aesthetic based entirely on what is the most comfortable for the human mind to consume, this will be a more stylized and interpretative aesthetic than a literal copy of physical reality. It's already doing this, I think it will keep going that direction.

I know some people may read this, perhaps think my assertions are interesting, but then notice it is entirely based on anecdotal observations and experiences, then wonder where is the white paper study? Which on that point, I completely agree. I wish I could be linking research concerning this and I really do wonder myself where is the research on this? But I don't write white papers, I only write code and spend a lot of time in VR. I am not directing the millions of dollars at various companies. Why are companies spending millions, billions, on the advancement and research of graphical techniques without comparable research on how those techniques and styles affect people in VR? My big contention is that people just simply assume certain aesthetics are desirable in VR without having any reason other than "It was desirable on a screen!" or "It looked cool for the 30 minutes I tried it!" Which I do not think is a good reason. There is so much research that could be done on this in a methodical and empirical manner. But until then, the best thing you can do is start spending many hours in VR yourself and pay attention. If a style or graphical technique worked well in VR, you should be able to make a world or avatar that utilizes that technique and becomes popular in VRChat right now to prove it.

Also, since I just recently saw the trailer to Nock I'd like to give a nod to the styling of that game. If I were to make an arena based game where people ran around I would probably make it similar to what the Nock developers have done, especially the darker night time arena. Choosing that styling for a game where users run around makes me think those developers have spent a good amount of time running around in VR themselves and paying attention to it. All surfaces are matte and untextured, the background is mostly a simple infinite-distance skybox, color palettes are simple, the worlds are not covered with many little objects creating higher frequency detail. I also notice only the ball has a hard-edge drop shadow resulting in fewer unnecessary hard-edged drop shadows moving around the world. Great visual design decisions for reducing discomfort in VR. It's not that I think all of VR should be this simple, just that it should be understood when designing experience and aesthetic, something like this should be your baseline and maintaining comfort should be the ideal over packing it with graphics for the sake of photorealism.