Thoughts on the "Super Goggle"

Introduction

Assumptions arise when one reads the AFWERX Super Goggle Digital Design Challenge. As I discuss this challenge, I will assume it was tailored for a system to be used in single pilot, fifth generation combat aircraft training. This assumption rules out multi-crew fixed wing aircraft, rotary wing aircraft,? joint fires, or any other role in air power. Narrowing to this use case is important to weigh the trade-offs to achieve the goals of this challenge. The laws of physics will not bend to our wills. However, I’d caution against the use of “super” if this is such a narrow solution. Other aviation specialties will, in time, want the same capabilities, but due to their subtle differences, different trade-offs are prioritized. You’ll find that the solution to the challenge is quite narrow.

A second assumption is that this challenge is for a permanent installation as the challenge mentions many cockpits operating in the same location. Deploying so many trainers at once is typically done at a specialized facility, normally not ad hoc for an exercise, nor temporary duty. Hardware being permanently installed factors into the settling of trade-offs I mentioned above. Break-down and transportability would be among them.

The challenge lays out separate qualities of concern, and I will address them in turn. I must discuss a more philosophical question before moving on to these categories, however: is the intent to satisfy a list of requirements or to satisfy the need for the best immersive device to use in a seated position? Before the advent of the Oculus Rift and HTC Vive, Virtual Reality was an obscure technology mostly used in Department of Defense (DoD) laboratories, universities, astronaut training, and the odd giant “five bucks a play” arcade. My first VR experience was a parachute descent trainer for the F-16 where you were suspended from a harness and did your best to steer into the wind, preparing yourself for impact with the ground. I found it to be an effective trainer even though what I saw was blocky and simple–the broad strokes came across! That device was tailored to specific requirements for that specialized use case. The headset chosen for the contract likely wasn’t of much use in other applications. Due to the low volume of these trainers across the DoD, it is likely this headset was ridiculously expensive. VR was not flourishing because of stringent, individual requirements. Engineers chased each wicket to satisfy the contracts resulting in headsets that while they met requirements, in their totality they were underwhelming. On the commercial side with the VR renaissance, however, all efforts were pointed to one principle: making great, affordable gaming VR hardware. Like this challenge, the assumption of the Oculus CV-1 was that it was to be a seated gaming experience. Trade-offs were made to design the best hardware possible, for that purpose.

OpenXR

In the early days of railroads there were competing “gauges”, the distances between the rails of the train tracks. Some of the logic in deciding how far to separate the wheelbase was engineering, but other parts of it were protective business decisions. Competing cars could not run on your tracks–they didn’t fit!? Over time, however, there were pressures to narrow down the gauges to down to a few.? This was driven by customers not willing to transfer loads to different railways. This was known as a “gauge break”.?

A similar line of events happened in the early days of the VR renaissance. Valve competed with Oculus on driving the OpenVR standard. Dynamics changed after the infusion of Meta money after Oculus was acquired, lowering the price per unit of XR hardware, while PCVR only solutions remained relatively expensive.

You may be reading this on a monitor or your phone. The video cable to your monitor has a standard socket, and your monitor displays my insightful words while adhering to industry display standards. The app or website on your phone is following standards and adhering to development languages to display this article as I intended to communicate.?

There’s a catch to strict standards adherence: what do you do when you innovate outside the norms? What if you discover a novel approach to dramatically increasing immersion, but the OpenXR standard doesn’t handle it? I’m working with a headset at the moment with an integrated depth sensor, but that depth sensor is unavailable when I leverage the OpenXR runtime in my software, for example. As I reflect on the target goals for this super goggle, I foresee many opportunities to use novel approaches to increase tracking precision.?

As I am writing this document, OpenXR 1.1 was released by the Khronos Group. With it comes many expected improvements. My argument of novel solutions and lagging standards stands.

Multiple Image Generator Support

At I/ITSEC 2023, I got my hands on a Lynx R-1. Before that moment, I had never seen one in person.? It was engineered to Open XR standards. The software I develop supports the OpenXR runtime. On the first try, almost everything worked! We made a very small tweak to our software to handle the custom passthrough color (black to green) for our table top battlespace viewer after some advice from Lynx engineers. UltraLeap hand controlling was already supported and worked as well. This is a great example of how adhering to standards yields interoperability! Remember, when you plug in a monitor to your PC, you assume it will just work. XR hardware is approaching this state.

The challenge requires multiple Image Generator (IG) support. It will always be easier for software to bend to firmware. I think the intention of this requirement is making the headset as easy as possible to integrate into an IG. It’s up to the IG to accommodate the hardware by either natively supporting or by writing plugins. It takes two to Tango, however. Having a robust software development kit rife with example code and documentation is the key to success.

In the past, before OpenXR, while working with other IG companies, when new XR hardware was introduced on the market we requested device support. Most of the time for integration was waiting on either party to receive the headset plus a little haggling to ensure this was something our customers were purchasing and worth the effort. In short order,? there was a new plugin “.dll” ready for testing. To summarize, this portion of the challenge is really for the IG, but the hardware manufacturer can make this easier and faster!? Another trade-off: you have to dedicate resources to solid documentation and design processes to achieve this lack of friction.?

Relying on gaming engines is a way to make this integration happen faster. If the headset manufacturer provides an SDK to 3D runtime engines: Unity, Godot, or Unreal with examples, this accelerates integration.

Perception of Time of Day

It’s amazing how the human brain can fill in the gaps, doing some of the heavy lifting for immersive technology, while simultaneously, a small missing detail can pull a user out of the illusion of presence! I’ve encountered this while flying in MR cockpits. The cold glow of fluorescent lights on my arms clash with the night sky. SDKs include solutions to this for gaming engines: it requires leveraging a custom shader on the Mixed Reality camera input. Refer to the section above. The headset manufacturer should provide examples for integration, but ultimately, it’s the IG’s responsibility to make it happen.? This requirement dovetails with the requirement for NVG emulation. This assumes that the HMD has a full facial interface, i.e, the user is unable to look down a physical gap below the lenses. That simple design choice has its own advantages. The trade off in an interface with peripheral vision is that you’ll need to rely on “mood” lighting as your surrounding vision remains un-augmented. There are many content aware LED lighting systems on the market. I use one on the rare occasion I play a “flat” game on my PC.?

Tracking

For XR hardware with six degrees of freedom (6DOF), meaning, as your body moves around the devices follow, as opposed to rotating around a point, there are two ways to track the movement of the devices: outside-in and inside-out (IOT) I’ve seen a third method by using onboard inertial measurement units (IMUs), but some drift over time without an moveable reference, and others that use magnetic reference are not as widely adopted as the other two methods. The most common method, available to consumers, for outside-in tracking is the use of Valve’s (HTC has now taken over production) Lighthouse. There are also professional grade options available like Polhemus, OptiTrack, Advanced Realtime Tracking, and many others. Each of them have their strengths and weaknesses and I will discuss this more below. IOT uses onboard cameras and computer vision to maintain 6DOF tracking. This is great for transportability and maintaining a small footprint. IOT, however, sometimes means more electronics hanging on the head, which will slightly impact comfort and wear time. IOT requires that the area and the reference points within do not change much. Changes in lighting, actions by other personnel and reflective surfaces may confuse the system resulting in pauses in tracking. This may contribute to simulation sickness. However, IOT tracking algorithms across the industry are improving at an amazing rate. Another leap forward is expected with event cameras otherwise known as neuromorphic cameras will be used. These cameras look for changes in their field of view, as opposed to feeding an image to be processed by computer vision each frame. Lastly, IOT must learn where it is the first time. This requires a calibration and centering process. The exception is visual odometry, where the headset locks onto a previously mapped space by seeing fiducial markers as known reference points. If for some reason, the environment changes or the computer vision is confused, the process must be done again. With outside-in tracking, if nothing changes about the computer, this procedure is only done once. Returning to our assumptions about the challenge: we will assume the desired solution is a permanent installation. Hence, outside-in tracking may be best. Furthermore, having inside out tracking is best for a rotation of pilots through the system. IOT headset may get confused if they are placed on the seat and can’t see a reference point. With inside-out tracking all references remain constant. If the cockpit does not have an adjustable seat and is fully virtual, with a press of a button the head can be “re-caged” to the desired position in VR.

If outside-in tracking is chosen there are some qualities that must be considered. I’ll discuss the Valve Lighthouse.? First, you need a stable structure for which to mount the Lighthouses. When deployed correctly, Lighthouses can maintain tracking accuracy of less than a millimeter! Multiple and different brand headsets can share the Lighthouses–you can put as many as the five meter squared area will support before humans and equipment physically block the beam! To have the best fix, Lighthouses should be deployed as far apart from the headset as possible: opposite corners, for example.? Some facilities with false floors introduce flex as people walk around. Facility managers often will not allow them to drill into the wall, so they are mounted on tripods. If the lighthouse moves even the slightest, the further the headset is away from the lighthouse, the more it will wobble! Other strategies have been to mount the lighthouse on the edges of the cockpit. Sixteen total Lighthouse V2’s can be in the same place without causing interference. Assuming you use two Lighthouses per cockpit, you can only deploy eight cockpits in one space. Their precision will be greater than having them mounted on walls further away, as there is less distance to introduce error, but fewer headsets can take advantage of the beams. The optimum solution for the Lighthouse tracking is wall mounted Lighthouses, enough to track all cockpits in the space, but allowing for instructors and maintenance personnel to walk around without interfering with tracking. This assumes more than eight fighter cockpits per room.?

The ART SMARTTRACK works by placing a device at the front of the cockpit. The HMD will have IR reflective spheres for reference. That is the only required tracking hardware to place on the HMD freeing up weight for wearing for long durations. If the challenge is to build the best fifth generation cockpit HMD, removing as much material as possible that must hang on the pilot’s head moving it to the cockpit is a winning strategy for simultaneously accomplishing many of the parts of this challenge.

If you have seen footage of actors in motion capture suits for video games and movies, you’ve probably seen the OptiTrack products. Polhemus has a long history in military simulation tracking and is used aboard aircraft for head tracking–admirable accomplishments.

Ease of Use

I’m wrestling with this portion of the challenge. Most in the USAF do not fly planes. Did you know that in the USAF the physical and medical requirements for an aviator are the same to become an astronaut? Our aviators are held to a very high standard and can lose the privilege of flying for their country for very seemingly minor reasons. Medical waivers to retain flight status are a laborious process. I don’t understand why the word “inclusivity” is invoked for some of the most elite roles in the USAF. The only things I can imagine this portion is for is to accommodate support personnel, distinguished visitors, and family days. Aviator's eyesight can be corrected to 20/20 vision–this is common as an officer ages throughout one’s career, but there are limitations.? The headset will likely be used multiple times by different pilots in a day. If some of those users wear their glasses, over time you roll the dice of the spectacles scratching the lenses. The challenge calls for high resolution–the smallest scratch can blemish that perfect image. The HMD should have diopter dials to adjust the HMD to the prescription of the wearer.? This will account for the highest percentile of pilots who must correct their vision. The remainder can have bespoke inserts fabricated. These inserts will not scratch the lenses. Anyone who has worn gas mask inserts will be familiar with this concept. Bottom line: this HMD is the most crucial part of the simulator. Those fancy cat eye glasses frames shouldn’t get anywhere near the business end of it! As I’ll later explain, per unit cost for the HMD will be astronomical to the taxpayer.

As previously discussed, electing to use an inside-out tracking method will speed up the donning/doffing process. The longest portion of the process should be measuring the interpupillary distance (IPD) for the wearer, taking seconds.

Field of View

Those who have been in the XR industry know better by now than to listen to a marketer talk about the field of view (FOV) of a headset. What we do now is look for the size of the optics and their lens type within the goggle. Then we watch XR YouTubers motion with their hands where their vision stops. Here is why we disregard the FOV number: you are putting a very optically precise device, a device that has an incredibly short throw of light to a very unique and squishy human face. IPD and where the eye relief is set (eye relief is how far the lenses are from the eyes) make evaluating the FOV to be a very subjective experience. In fact, companies like Bigscreen and their Beyond headset require a face scan before purchase. The customer gets a personalized interface for their headset to maximize the overall experience while minimizing the size of the goggles.?

Here is the bargain: the more you increase your FOV, the less pixel density you’ll have. This was the trick of the HP Reverb 2. It had an amazing high pixel density and panel resolution, but it suffered from a very tight FOV to pack all those pixels together. If you want to read the dials in the cockpit or focus on an aircraft far away, but you don’t really care about looking for air traffic at the edge of your vision, that is, unless you move your neck to scan for it, the HP Reverb 2 was a good, affordable fit for flight simulator enthusiasts.? If you want a higher FOV and a higher pixel density, you must have more pixels in total. To make this successful you must have a power PC and an IG optimized for that load on the Graphics Processing Unit (GPU). Then you need to be able to move those pixels from the PC to the HMD in a timely manner! Any delay causes illness! Another trick to the bargain is to heavily rely on foveated rendering and eye tracking. The human brain processes peripheral vision with lower precision–you don’t need dramatically high resolution at the edge of your vision. It’s a waste of resources.? Render the entire area at a “lower” resolution, but while tracking the focus of the user’s eyes, make that area (the fovea) have the highest amount of pixels as possible.

Curved display panels for use in XR headsets may be more common in the future, but for now, they are flat.? I’ll set aside all the different panel technology available, pentile configuration, etc. To get a large FOV the light from those driving panels is bent by the lenses. In some cases, this brings distortion, which can be accounted for, but most importantly it brings two things: weight and moment.? Moment means that even if the hardware is quite light, it hangs far from the face and levers against it. Quick head movements will make the HMD feel like it will fall off the face. Over time, pressure points will develop making it uncomfortable.. The headset must be as close to the face as possible. Again, offloading electronics off the HMD onto the cockpit rails will free up weight and space.

Resolution

As discussed, there is a relationship between pixel density and FOV. The state of the art and industry choice to drive as many pixels as quickly as possible from a GPU to a display is corded DisplayPort. An immersive image requires more than just the headset–there is a powerful PC and GPU solution whipping up those pixels. In order to satisfy this challenge, this portion of the solution cannot be ignored!

When you have used corded HMDs for a long time, you recognize what an immersion breaker it can be when you no longer have it! Wireless PCVR is done by compressing the rendered video stream and sending it to the headset over a high speed, high bandwidth WiFi router. These All-In-One (AIO) headsets are, fundamentally, just highly customized Android phones strapped to your face. There are power and graphics limitations to what they can do, but their portability and being self-contained products have their appeal to a mass market who would rather not tinker with a connection to a PC. There have been advances in upscaling along with compression. With compression, however, artifacts may appear. If high resolution, meaning the most amount of pixels driven per second possible, this solution will have to be corded. The trade off for comfort will necessitate a cord. The user is seated. In a cockpit, the pilot is connected to the plane through all sorts of tubes and cables. The cable could be managed along with these.

To bring this together: FOV, resolution, and comfort, you are forced to decide on trade-offs, a foot stomping theme of this document. Foveated rendering lets you cheat, a little, but that comes with its own particular penalties too: more optics and electronics weight. I don’t think the 8K requirement and its caveats is the best way to approach this, nor do I think the honestly impossible to measure 120 degree by 105 degrees stat is a good benchmark too. Remember, YouTubers holding out their arms is, frankly, more scientific.

Rather, I’d focus on what you would like to accomplish. For a fifth generation fighter, I would propose the following drill to judge the criteria altogether: be able to see information about a target on the panoramic display. After clearly seeing the information, be able to move the head to where the expected target may be in front of the aircraft. This must be done with no noticeable tug nor discomfort on the head and neck from the HMD. Within a reasonable amount of time, the pilot must be able to see the aircraft at an expected distance and confirm the aspect given to him from the panoramic display. This is where the IG matters! Many IG’s use a scale factor to artificially make the aircraft larger the further away it is.. This is so that on non-XR displays, when an aircraft is at the distance away where it should be visible. A full high-definition display, with limited pixels, will at least light up a few so that it can be seen. You are swiftly moving the camera which means that the engine may need to load assets such as terrain and the model of the aircrafts. It may need to reassess the sunlight on clouds you are just now seeing. Say instead of following this drill, you decide to check for icing on the leading edge of your wings–the system has to handle this quick movement and make the image happen clearly and without inducing sickness. Until you had decided to snap your head over your shoulder–the system may not have allocated resources to draw that for you, or maybe it did with a light weight method and does something if the condition changes..? Good software does its best to handle these things without the user ever noticing–if all these things happen smoothly, regardless of the headset of choice, the developer is earning their pay.? Motion to photons matter in XR! The HMD must be fast enough to move from the panoramic display, which we will assume to be a Mixed Reality cutout, to the target far away. The challenge has not mentioned focal depth: focusing on a near object for readability to far away objects.? The ability to quickly shift focus should be considered in the challenge’s design criteria.

Comfort

No one enjoys wearing something heavy on their head, especially for a long period of time. I would claim that as axiomatic. The aviators for whom this challenge is designed are expected to wear Helmet Mounted Cueing System (HMCS) in flight. Moreover, they experience “G” forces as they maneuver the aircraft, multiplying this weight. This equipment is its own HMD of sorts already.? With in-flight refueling that duration where they endure this may extend further than the three hours in this challenge. Let’s make yet another assumption: you can learn when you are challenged, but learning diminishes when you are in pain.

I have seen simulators be a bit more relaxed, however. The pilot wears aviation headphones (David Clarks or Bose, for example) Either they bring their set in or they use the ones stored on site with the simulator. Yes, here’s another trade-off.

To match the inflight experience as much as possible, the HMD would be mounted on the helmet. There are many examples of the Night Vision Goggle (NVG) mount being used to clasp the equipment on the helmet. The goggles can then be swiveled up and away when not being used. When designing the HMD, the goal should be to emulate the HMCS weight and moment as much as possible. This is not just to best simulate the in-flight sensation of moving one’s head, but leveraging all the testing and evaluation of flight equipment.

Going with this route introduces some logistical concerns: flight helmets are customized by the Life Support shop in their unit for each aviator. The oxygen mask is properly adjusted to avoid leaks, and the padding is adjusted with care to avoid hot spots to the wearer. (My jaw joints ache and pop thinking about how long I wore my helmet and mask and that was customized for me!) This would mean the pilot needs a sim helmet too. The Life Support shop won’t be excited about doubling the workload.?

Let’s assume helmets aren’t going to work. As mentioned, we need to assume that the cockpit and HMD will be used by multiple users throughout the duty day. It’s best to wear aviation headphones to handle communications. They are easily adjustable and easy to sanitize for the next wearer. This will also allow for a counterbalance to be incorporated into the HMD superstructure to better center the weight to the middle of the skull, if desired..

Security Considerations

Despite all the resources in research laboratories, partnerships with academic institutions, and the assistance of professional organizations, with very few notable exceptions, the DoD overall has a poor record of integrating Consumer Off The Shelf (COTS) XR hardware into training programs, particularly those which require protection of classified material (IL5 and above) since the dawn of the VR renaissance. You know? With all the research programs, SBIRs, internships..the heck am I ten pages into this document when the DoD has all these resources?

The success of the Pilot Training Next initiative was due in part to physically moving the program to Austin, Texas, out of the grasp of professional naysayer staff of cybersecurity personnel so they wouldn’t tank it. The Risk Management Framework (RMF), Certificate to Field (CtF), Authority to Operate (ATO) processes are entirely broken and unable to even begin to cogitate XR technology. Valiant attempts are made to create “hacks” to the laws, set up offices, finding novel ways to loophole the acquisition process. Some have tried to bundle various training XR programs under one umbrella to provide a unified front to the human speedbumps. The fundamental fact remains: the whole damned thing incentivizes saying “no” and people make more money the more they say “no”. They get fired if they say “yes” and there is a breach.

So, where are we now? John Carmack, the former CTO of Meta’s Oculus, got into the VR fray because headsets were tens of thousands of dollars per unit and thought we could all do better.? We are right back to that now as Varjo has tried its best to create a product for which the US government has been less than forthwith on defining the requirements. Each individual system undergoes this arcane and lengthy process either for a type (God forbid you upgrade anything) or for a location. One unit may be approved for classified use of XR, the other with the same hardware and software might not! It gets worse! I just mentioned two identical units, both will wait for each unit to complete their ATO saga so the other doesn’t have to do the work leveraging a rule of reciprocity, so nothing gets done at all!

The writer of the Security Considerations portion got it right. Offline is crucial. Never should the device attempt an upload to the mothership! You will find, however, wanting to remain offline is a very foreign concept to those in the XR industry. Let’s be honest, many see XR as surveillance devices to sell ads–it’s perfect for the role. Thwarting that is a threat to their entire business model!

Ideally, nothing is stored on the headset. When power is removed and all the electrons drain, there’s nothing to glean from the device. We’ve already established that it needs to be corded to a powerful PC–that’s where the data is. This means that the HMD can leave the facility to be serviced or repaired. On base repair is ideal to ensure chain of custody of the device. I fully support having it being field repairable. This won’t be easy. HMD’s are paragons of precision. Anything out of alignment and you’ll see it or feel it. Making a unit field repairable will also make it bulkier so you can safely remove and reinstall parts. Take a look at your sleek smartphone–a whole lot of glue holding that thin thing together.

I urge the DoD to not be dogmatic on RF propagation with XR headsets. Not all ‘trons are the same. Secure WiFi exists. RF absorbing paint exists. EM propagation is a measurable quality. There is capability you are leaving on the table otherwise. Say through a technological advance the HMD can go to an All-In-One (AIO) profile when the cable is detached, all other trade-offs sorted. The pilot can step in and out of the cockpit and perform a mission brief and debrief with the headset on. A lot of briefing time is talking to a white board and gesturing. Take advantage of the technology and speed up these portions of the mission! I bet a patch wearer can trim his debrief to three hours with this tech! (inside joke)? As COTS XR continues to evolve, the DoD may find very few offerings with a cable–they’ll practically all be wireless in time. If an AIO somehow makes the grade, it will need to be able to be “zeroized”. Best to hope for is a full reset to a default configuration.

We need to talk about China.

Current XR hardware is a testament to the levels of global interdependence we have reached. Without robust global trade, this achievement would be highly unlikely. We know very well the first image this new headset will render against the sky is a Chinese fighter. It’s disingenuous for the US to train for this ordeal when the tag says “Made in China” on it., even if the individual parts have been reassembled elsewhere.

However, Stan Larroque, CEO of Lynx said it well: “Sovereignty in semiconductors today doesn’t really exist in our world” when laying out his actions to avoid conflict areas in the creation of his product.? Somewhere down the line, I imagine it is difficult to find any example of XR hardware that hasn’t been touched by China in some way. While chip fabrication is breaking ground in Florida, the US is presently woefully behind Asia. The TAA and BAA requirement is a noble goal, but that incurs expense. This is one reason why this headset will be expensive.

There are many calling for a “decoupling” with China’s economy and ending the US’s dependence on its resources. Easier said than done, and won’t be done before delivery of this headset. The best that can be done is to head off any possibilities the HMD can spy for the CCP by strictly controlling components that could possibly make that happen. Making a headset that does not store data heads off almost all of those worries.

https://www.youtube.com/live/VRM9TYQpqA4?si=Lns-wcdzdVkGp5WB&t=2099

Economy of Scale

Two routes lay before you. You can hope that a boutique company can design, fabricate, and support this bespoke hardware, or you can hope that a large volume HMD manufacturer will create a modified SKU of an existing headset for your challenge. You guessed it..trade-offs.

If you go with a smaller boutique HMD manufacturer they’ll try with all their IP to create a product to your specifications. There is no behemoth of corporate culture to row against trying to get this challenge to become a reality. There are no platoons of lawyers trying to take advantage of good will with the AFWERX team on which documents can be released to the government, nor strong arming marketing opportunities.? Here’s the rub: they don’t have the giant stack of capital to throw money at the most challenging problems in XR. They don’t have the ability to dip into a bucket of intellectual property that, while research prototypes may not have panned out for a consumer headset, are perfect to leverage for this challenge. The boutique solution won’t have an entire floor dedicated to product testing with peculiar robots emulating hard human use. My experience has shown when you get a boutique simulation device you always treat it as incredibly fragile. Now, how many of you almost sat on your kid’s Meta Quest while reading this?? If you go the boutique route, you are going to have to supply them with enough capital to keep their doors open and to service this headset. Then you are going to need to give them more because counting on the DoD on this project alone to keep the doors open is a horrible business strategy! Don’t be one bit surprised if the boutique manufacturer immediately looks for other buyers for this product vying for some stability! This is where I come back to a point made in the introduction: you want to make something for fifth generation fighter pilots–in about the time it takes for a two circle merge, someone not a fighter pilot will be wanting this headset, but with changes for the dynamics of their aircraft!??

The solution is going to have growing pains. XR can be amazing, but there are headwinds that make engineering difficult, namely the spatial nature of the technology. The PC I’m writing this document on isn’t precariously strapped to my head swinging about many degrees per second in six degrees of freedom. It’s been sitting on the floor of my office and desperately needs to be dusted.

Going with the large manufacturer route brings its own problems. You are asking them to modify a product line that took a mountain of internal research and development to create. Some circuit boards are highly optimized for weight: the transceivers for Bluetooth, controller tracking, and WiFi are not removable without damaging the device. The best you can hope for is firmware to disable these transceivers, but you just paid a lot of money for a broken product! Golf clap: great use of taxpayer money. This is another reason why I beseech the DoD to reconsider how it handles RF in classified areas. You are essentially asking the industry to make you something with vacuum tubes. Evolve, mitigate, or fail.??

Look no further than the HTC Vive Pro Secure. The technology is vintage 2018. It’s still a good headset, but the technology has come a long way. Notice how their website doesn’t show an HTC Vive Pro 2 Secure? I’m not privy to HTC’s business strategies, but it appears to me like it wasn’t worth the effort for HTC to accommodate the DoD’s Ludditism. There’s just not enough volume to deliver to make a profit. HTC has products tailored for enterprise use and location based entertainment (LBE). LBE can effortlessly be leveraged for military training in areas well beyond this challenge. The moment you try to rise above IL4, however, they naysayers start lying down in front of your tires.? I’ve stood in a space cleared for classified use with a headset on. A wire trailed from my neck and I tried my best not to stand on the wires going to the hand controllers. All I could think was: this is stupid. To quote Carmack, “We could do a lot better.”

Let’s look at Varjo’s strategy as I see it. The price per unit attempts to account for the low volume they might ship. I see them as the Leica of XR hardware, when you want to enjoy the hand stitched leather on your camera. They have options for classified use, as best as they can guess what that may be, but this comes with a premium. Some of my customers are individual units. They come to me knowing the price of a Meta Quest 3. Then I tell them the price of a Secure, TAA compliant, Focal Edition XR-4 with SteamVR tracking.? An amazing solution for sure, but it’s not within the budget for individual units. The XR-4 was not specifically designed for a fifth generation fighter pilot. They tried their best to include that in their design as their product is offered to as many use cases as possible to remain viable. They know that there is a dedicated “simpit” customer base to buy their prosumer product. Sales to this base covers the fickleness of enterprise customers dabbling in XR to reduce costs by investing in its use. But here you are, asking for a bespoke version with these challenge requirements. The cost per unit will soar faster than a slick F-15.

We then have firms that ride the edges of these two paths. There’s a common phrase in the XR industry: “hardware is hard”. Global trade enabled the rise of consumer XR hardware, but global regulation is trying its best to keep the “little guys” out. Those trying to be more than a boutique XR hardware provider have to rely on crowdfunding and late night dinners with Russian oligarchs for crumbs of funding. This challenge would be a windfall if they won to avoid that. But they have their own particular problems. They find themselves blowing a portion of their capital buying a robot for the Taiwanese vendor, setting you back years on delivery. A Malaysian box printing company gets the wrong ink, bringing another shipping delay. The CEO’s of these companies wish they had fleets of black helicopters air dropping lawyers to get the product to ship. Meanwhile, Zuckerburg takes another swim in his Scrooge McDuck vault coming back to whisper in his lavalier microphone like the Zoomer he is not.

Conclusion

This challenge is an attempt to return back to the days before the VR renaissance, where XR hardware was about ticking individual contractual requirements instead of holistically creating a good experience across a wide range of use cases. The DoD needs to come to terms with this sad reality: it’s no longer about you anymore. The DoD no longer drives innovation in immersive technology. The best the DoD can hope for is making players in the XR industry look better for other verticals. Worried about your new car design being poached by a competitor? This headset has been certified by the US government to not leak state secrets. If it’s good enough for paranoid Uncle Sam, it’s good enough to proceed with cutting virtual clay with hand controllers. The price per unit needs to be low enough for “simpit” enthusiasts to know that they are wearing the same headset real fighter pilots wear when they train, because whoever makes this headset for you can’t keep their doors open otherwise. I’ve participated in an AFWERX challenge before. One thing that I really didn’t like was the vagueness of funding. You are asking the industry to take risks on your ventures, losing the house, car, and wife risks. You are posing this challenge without revealing the true prize. Again, it’s no longer about you. These businesses no longer rely on DoD alone. Don’t be surprised if the larger manufacturers approach your challenge with skepticism and with as little effort as they can afford, and the smaller manufacturers pray you don’t see the capabilities they have yet to achieve. Also, don’t be surprised that in time no one will be willing to create what you want. The market for nixie tube clocks, while interesting, isn’t a flourishing one. As an XR professional, I’m not going to deal in vacuum tubes any longer.

This document contained the opinions of a crusty, well-meaning, patriotic simulation engineer who has installed a wide range of XR solutions with a wide range of customers. I don’t claim to know everything, nor do I claim to be correct on everything I’ve discussed. I hope this document colors how the AFWERX staff approaches this challenge. I install XR solutions with the headsets the customer has chosen. This was a rare opportunity to weigh in on what I would like to install for my customers with all the saltiness I accumulated over the decade. I tell people I’m not in the arms industry; I’m in the sword sharpening business. I hope my words help in achieving the best training technologically possible to our warriors in facing their enemies without fear nor with hesitation.?