HMI Challenges on the Different Levels of Autonomous Driving: Thoughts of an HMI Guru

This article is a shortened version of a white paper I wrote. If you are interested in the complete paper let me know!

Humans are meant to be mobile. Our ancestors where stone age hunters and collectors, mobility was core for them to gain the required amount of food. An average person at that time was mobile for at least a few hours per day. The entire history of mankind is permeated with mobility.

Today we face growing urbanization. Megacities with more than 10 million citizens are located in almost all regions. Traffic congestion is standard, long commuting times normal. Connected and autonomous vehicles are one solution for this. The more drivers are out of the driving task the better the commuting times may be used for alternative activities.

Traffic will become safer with more autonomy in driving. For most of the driving situations and the majority of decision making a well-developed and implemented technology will be better than humans with their specific strengths and weaknesses. On the other hand, we as humans in the technological world are still somehow strange with the artificial prothesis we built and use.

Level 0/1: Today’s Driving

Tons of literature have been written about Human Factors in vehicles, car and driver interaction, about driver distraction, information overload, and driver fatigue. They describe two phenomenons we all experience when on the road: overload and underload. Since the focus of this post is on autonomous driving, no detailed analysis will be made. An up to date overview is given for example in Akamatsu (2019) and Walker et al. (2015).

Level 2: Who is doing What?

On level 2 we have a shared driving task. Some parts are performed by the machine, some by the human. In some use cases the switch between the two instances happens frequently. Plus, that many existing level 2 cars force the driver to intervene regularly, for example by rattling the steering wheel every 30 to 60 seconds.

Three major HMI issues occur on level 2:

1. Humans tend to pull themselves out of a task they don’t like to perform or that is perceived as boring. Misuse of the technology will lead to critical situations when drivers cheat the system
2. The continuous and basically unpredictable hand over between driver and car leads to stress
3. The vehicle needs to communicate, which task is to be performed by whom. If the car thinks “I’m driving” and the driver thinks the car is driving, we have an autonomous mode. The other way around is manual driving. If both think, they should drive, the driver is driving, since the automotive industry has the paradigm, that the human always overrides the machine. If human and machine think, the other is driving the situation gets critical, since no one is driving. See picture below. The drive mode, the functionality needs to be communicated clearly to the user. The HMI needs to indicate doubt free, which task the machine is performing, and which task remains for the driver. Most of today’s solution lack this clear communication

Level 3: The Bottleneck

On level 3 the driving task is shared like on level 2, so we basically face the same issues as mentioned above. Level 3 cars allow the driver to leave the constant control loop, meaning that they can use their mental and visual capacities to perform other tasks. This may be reading, playing games, or surfing in the internet. When the technology detects problems, it cannot solve by itself the driver needs to take over the driving task. The time frame discussed for the takeover is a few seconds.

Various studies (see for example Damb?ck et al, 2012, Merat et al., 2014, Petermann-Stock, 2015, Vogelpohl et al., 2016) indicate that the time required to perform a basic handover is between 3 and 10 seconds, depending on the complexity of the driving task and the secondary task. A full handover needs up to 40 seconds, some sources say 120 seconds.

To allow safe driving a full situation awareness is required. This includes the cognition of objects in the environment, the meaning of the relevant parameters and objects, and a correct forecast of the relevant near future, which is in driving about 10 seconds.

The big issue on autonomous driving level 3 is the time the driver needs to build a complete and correct situation awareness. With certain HMI measures it may be possible to shrink the time required. Level 3 may kill the entire autonomous driving story.

Level 4: Loss of Competence

On level 4 driver activities are very seldom. The car performs in almost any driving situation autonomously. The HMI issues mentioned for levels 2 and 3 may occur but have a lower relevance.

The driver will experience a loss in competence. To solve the issue the car may force the driver to drive a certain amount of time manually, for example 1 hour per week. This is done in the aviation industry, to keep their skills alive pilots need to fly manually a certain amount of time. For a professional pilot this is an appropriate measure, it remains open, if this is valid for ordinary car drivers.

Level 5: The 25th Hour Spend Motion Sick?

On level 5 all driving situations are solved by the vehicle, no external intervention is required at any time. The responsibility for the driving task is not on any person inside the vehicle. Besides technological issues, the legal situation around responsibility for the car is an open issue.

A few HMI issues occur on level 5. One is motion sickness, which may also be relevant on other levels of autonomy but may be more obvious on fully autonomous cars. User will want to perform other activities in self-driving vehicles: reading, surfing, working, shopping, communication. The time we gain because we do not need to control a vehicle in a traffic jam is often called the 25th hour, an add on to our lives.

A worst-case scenario will be users wanting to be entertained while on the road that cannot enjoy the time on the road. Different studies show different relevance of motion sickness. Today people read in cars when the are passengers, in trains, or on busses. Motion sickness occurs more often in cars due to higher ranges lateral and longitudinal forces. Level 5 vehicles may have a limited range in acceleration and lower curve speeds.

Other HMI issue on level 5 are how to book and call a car, how to communicate the destination, how to communicate stop overs and route changes, how to set the climate, how use the common space in the car when on shared rides.

_____

On the author:

Dr. Peter R?ssger. Keynote Speaker. Consultant. Coach. HMI Guru. HMI Expert. HMI Punk. Usability. User experience. Human-technology interaction. We need to know everything about humans in order to shape technology. And not the other way around.

I adapt technology to the skills, needs and desires of people. And not the other way around. With my keynote speeches, reports, consultations and coaching, I take my clients from the automotive and high tech industries to a new level of knowledge, action and decision. Make technology sexy!