Eye-Tracking Technology Innovation in 2022: Your Top 10 Questions Answered

This a podcast transcript with innovation expert Matt Wahlrab of Rapid Alpha answering questions about eye-tracking technology.        

Q: What is your experience with gaze detection technology innovation?

I have been working on innovation with gaze detection technology for over a decade now at this point. What's really exciting about this space is how the technology is rapidly evolving and how we're starting to see the latest advancements in the news. The fundamental technology of gaze detection is benefiting from dramatic improvements in the ability to accurately detect what someone's looking at without expensive cameras and head mounted sensors.

We saw early pioneers that were in the space and their early patent filings as foundational to growing early markets in accessibility and the video gaming industry. With the recent advancements over the past 5 years, these early devices seem rudimentary with their need for frequent recalibration and large head mount systems. But we don’t see retail, medical applications, and the demand in consumer electronics without their work. After years of many players making incremental changes to near infrared (NIR), it is exciting to see new solutions making mid-range and long-range applications, those that don’t require a head mounted system take off.

In the mid-nineties when the earliest technology emerged, NIR was used to irradiate the eye, which required hardware be worn on your head or on a monitor about arms length away to pick up the NIR as it emerged from the user’s pupils. By mounting the hardware on something like glasses, it reduced the need to recalibrate to find the user’s eyes. Due to processing speeds, sensor sensitivity, environmental noise, and system costs, head mounted really helped simplify the challenge of determining what a user was looking at.

There weren't a lot of the use cases that are possible today gaining traction in the 90’s and early 2000’s. The split of Tobii into two companies to focus on different segments highlights the challenges facing early pioneers. These devices, priced from $17,000 to $20,000 are highly accurate but generally find traction in more niche applications. Contrast this with relative newcomers like Oculus, priced for around $300 dollars, which are going after growing consumer markets. Still, in the macro, to gain widespread adoption and capitalize on the technology, there is an unmet need to move detection systems from the head and onto displays if calibration challenges can be solved.

Today there are solutions that are benefitting from computer vision and forward-facing cameras on devices like mobile phones and tablets. The ability to deploy on high-volume devices and older generation technology is opening the door to mid- and long-range use. Industries like brick-and-mortar retail, where consumers continue to crave novel experiences, have an opportunity to compete against the allure of e-commerce.

We had to work through that technology evolution process to get where we are. With the vision described by Metaverse and their subsidiary Oculus they're really going to need things like gaze detection to make the devices work well in a cost-effective platform. But whether the experience Meta promotes will be met with enthusiasm of younger generations when older demographics have fewer touch points leading to adoption, remains to be seen.

It’s that classic crossing the chasm with innovators and early adopters, and then moving through to the gateway adopters to mass adoption.

Q: What is gaze detection technology?

Gaze detection technology dynamically determines the location of a user’s eyes and direction of a user’s gaze. The location of the user’s eyes and direction of gaze is used to infer an area of focus, say on a screen.

Traditional gaze detection is worn on the face where NIR light shines into the eye from camera systems worn on the inside rims of glasses. Since the camera is worn, the distance to the eye is “known” and generally remains constant (at least unless the user adjusts the glasses). By determining how much NIR is received into the eye and reflected onto the camera, a person’s direction of gaze can be determined. Unfortunately, NIR can be washed out by natural sunlight, making it hard to apply this technique when the distance between the user’s eye varies and camera vary. Another complication occurs in that a person holding an iPad for example or sitting on a sofa viewing a TV can move the iPad in their hands, scoot up on a sofa, or just change the position of their head. Many systems and open-source software would require a user re-calibration to detect the user’s eyes and gaze direction.

Today there are a several applications in which NIR isn’t practical. And this gives us an opportunity to talk about where gaze detection is today. Namely gaze detection in mid-range and long-range applications using visible light.

Q: Where do you think gaze detection technology innovation will go?

From improving prescription testing and quality of diagnosis in optometry, to controlling a screen hands-free in a retail space (no need to touch a public use screen or pick up germs), or use cases in automotive (control your radio, update directions, make a phone call - all without ever looking away from the road). There are in fact many use cases for gaze detection.

People are talking about e-commerce and e-commerce eating brick-and-mortar retail. I typically think of this in terms of the competition in the USA between Walmart and Amazon. Location-based retail really needs something to differentiate itself against the e-commerce giants. Large retailers understand and appreciate the need to create these immersive environments for their customers because people are physically there.

If we want to continue to sell things well in location-based stores, customer experience is important.

Thinking about the ways that retailers can interact with your clientele and enrich their experience, one of the things that can really differentiate and create a new and compelling customer experience is gaze detection. How can retailers understand what people are looking at, especially in a post-COVID world where people are worried about touching screens?

For example, customers will interact with things at a distance or observe them for purchase consideration. That creates some very interesting opportunities to differentiate from e-commerce, and to create a better in-store retail experience for the customer.

Once the customer is in the store, that's one of the areas where there are opportunities for gaze detection applications. ?

Q: What are the cutting-edge opportunities?

Other growth areas are in gaming and in ophthalmology by extending innovative gaze detection technology. Certainly, in ophthalmology, there are lots and lots of eye scans. With new data and new gaze detection capture technology that may unlock new diagnosis for eyes and possibly even new treatments arising from the data.

Automotive is another growth area. Automotive engineers are interested in being able to determine what the drive and vehicle occupants are looking at. One of the main the challenges with respect to automotive is the steering wheel. The way that steering wheels are configured today can conflict with the placement of gaze detection cameras and sensors, and for obvious safety reasons there are additional regulatory hurdles. But over time the safety advantages will prevail, and the auto industry and consumer will embrace the benefits gaze detection, especially in an environment that's rapidly changing and making sure right that driver is focused. Also, there are more interactive in-vehicle experiences, like adding hands-free control of environmental and entertainment controls.

Q: What are some of the challenges with the current state of the technology?

Getting the technology to work and overcome some of the background and environment challenges is at the top of the list. For example, with an at-home entertainment device, there may be several family members (and pets) watching the screen. Data collection becomes very complex.

If the user is not wearing the gaze detection device, there’s separation in three-dimensional space as to where the subject is seated or leaning back, even calculation of the head position can be a challenge. Or if the subject is moving, how does the technology determine what the subject is looking at as it relates to the screen?

Then let’s add in a child's face based upon age, size and shape of the eyes, demographics, race, etc., all these calculations must happen as the user’s gaze direction changes and updates with head movement. These rapid calibrations have been a barrier for widespread use.

There is exciting new technology coming out that helps for low light situations, and it can detect what someone's looking at, regardless of the user demographics. The innovative gaze detection technology can rapidly compensate for distance from the camera and multiple users looking at the same screen without having to do a cumbersome or intrusive calibration, even as new users come into the field of view.?

If the technology must recalibrate every movement of the subject’s head, that's not going to be a great user experience. There is innovative emerging technology that indicates there are breakthroughs to overcome those challenges.

Q: What's a state-of-the-art leading gaze detection technology?

What's going to solve a relevant problem? If you want to have a consumer grade electronic device, like a cell phone, a tablet, or a TV with gaze detection capabilities, the technology has to be able to overcome the calibration issue. That’s a combination of processing power, data capture hardware (ex. cameras and sensor) and software.

Calibration must be seamless and dynamic. There are some fantastic breakthroughs that are happening. The most versatile and robust solutions are those being developed by Blink.

For example, overcoming low lighting to still capture eye movement. That's very exciting because it opens the opportunity to do more innovation with displays. With a 3D display, for example, to be able to render the right things and the right locations at the right time, depending upon what someone's orientation is in a seamless glitch-free way without changing the experience of what the user is viewing.?That’s an emerging and exciting innovation in gaze detection.

The most advanced gaze detection technology uses under 20 millisecond processing times as a benchmark.?I’m excited to see that performance level - there’s at least one company out there that is a sub 20 milliseconds for that calibration. It’s impressive performance for gaze detection. And because if this, there are new consumer experiences and new business models that are enabled. True innovators are working today to deliver high quality experiences to capture these markets first.

Also, there are advancements happening in marketing based on potential unlocked by gaze detection. There are a few companies that are starting to specialize in immersive brand environment, like Creative Labs. That’s best described as the experience from walking up to the store and creating a certain type of experience in anticipation for what it's going to be like once you're in the store. Then there's the ability, once you're in the store, to actually fine-tune the experience from discovery.

This innovative technology can help a retailer gain someone's attention by knowing that their eyes are in fact focused on the message, logo or content offered by their brand. That can then help a potential new customer discover new products, new features, new use cases, and then help them to explore in-depth something that the customer might be able to hold in their hand.

Q: What are the privacy concerns?

Gaze detection has the potential to be intrusive: undeniably.?Gaze detection technology will likely have some type of camera or sensor that's observing the user. For personal privacy, one innovation is for the gaze detection to take out all the pixels other than the eyes.

It’s like the old cartoons when somebody turned out the light: and only eyes were visible on a background. With this development in gaze detection technology, you will just see the eyes, this is where a lot of the is going, right, to be able to erase that type of PII data.?These are the sort of challenges the technologists in this area are tackling. They are conscious of the privacy issues and are actively seeking palatable alternatives.

Q: Given the prevalence of gaze detection technology and all the practical applications to stay current on innovation, is this something a company should create on their own?

There is appetite for open-source software solutions. Those options exist. But they are best suited for expensive cameras that drive up the cost of hardware when most applications are trying to bring costs down. Still, it is a useful resource to do proof of concepts, and pilots. But using such open source doesn’t provide any competitive advantage.

There’s also the emergence of SDKs (software development kits), where people can license 2022 state of the technology. Rather than spinning up a division to create gaze detection technology outside the business’ core competencies. That’s a preferable alternative. Working with an advanced gaze detection technology developer also makes sense because skilled talent in this area is hard to find.

There’s a dichotomy where you must have the technology to stay competitive and offer state-of-the-art user experiences using gaze detection technology. There's relatively low competitive advantage in using open source but partnering with the right industry-leading technologist can provide distinct advantages for your products and services.

Ideally, as a manufacturer of 3D displays (for example), partnering with an advanced gaze detection software/technology provider can provide that leading edge advantage for 3D displays. There’s a winning combination where the device manufacture can manage long-term overhead and costs. Then you will have industry leading experts with an innovative product roadmap to keep your devices leading edge too, and an escalation path to highly skilled technologists for support questions and issues

Q: Where are you seeing innovation investment in gaze detection?

A lot of money is being spent on reducing the complexity of wearables for gaze detection. Tracking gaze detection for multiple viewers simultaneously, operating in any lighting conditions, and accommodating new users without training are hurdles where progress is being made.

Q: What, in gaze detection technology, are you looking forward to?

There's so much happening in terms of content investment by Disney, Netflix, and Amazon Prime producing their own materials. What is exciting for me is what people will do in terms of immersive environments and 3D displays when viewing new content.?Innovation in this area is attracting investment -- it is coming and it is coming fast.

I fully expect that we'll be seeing more in this space, both for consumer electronics, as well as more commercial applications.

--

Matthew Wahlrab is an innovation expert and IAM Top 300 Global Strategist of 2022 recipient. You can email: [email protected]

?#Ophthalmology #Gaming #automotive #gazedetection #eyetracking #medtech #retail #3Ddisplay #holography #innovation #consumer #electronics