The Waiting is the Hardest Part

By Jim Blakley -- Associate Director, Living Edge Lab, Carnegie Mellon University

Note: This blog is a repost of the original at the Open Edge Computing Initiative

(c) 2021 Open Edge Computing Initiative

Like so many of us, I recently needed to search online for a Covid 19 vaccine appointment. I found a site that did a particularly good job of identifying locations with open appointment slots near me and enthusiastically began a frantic cycle of refreshing, entering my information and requesting the appointments. However, I soon learned that not only were others just a bit faster than I at snagging the golden tickets, but the system’s data update speed lagged the actual bookings of the appointments. I missed appointment after appointment. While I was frustrated with my lack of success, I didn’t give up – I just adapted my re-attempt rate to something more aligned to the real availability of new appointments. Rather than refresh every 30 seconds, I’d wait 5 minutes then 10 then 20 to retry. In short, the system taught me to slow my pace to its speed.

This pacing effect in response to system delays is both intuitive and common. And, it works both ways. In the early days of both Windows and Netflix, it was quite common to see a spinning icon that indicated that the system was slow. We learned to wait or to try something else. But, now, as these systems have gotten better, we have less patience for delays – we learned a faster pace. There is a long history of research on the impacts of system delay on human responses. As technologies and applications evolve, our understanding of these impacts also evolves.

Some recent work by Manuel Olguin Mu?oz, Roberta Klatzky, Junjue Wang, Padmanabhan Pillai, Mahadev Satyanarayanan, and James Gross at the Carnegie Mellon University Living Edge Lab and the KTH Royal Institute of Technology in Stockholm looked at these effects in the context of wearable cognitive assistance (WCA) applications, a class of augmented reality applications that I’ve discussed in previous blogs. These applications focus on giving real time automated instructions to a user performing a task, say, playing ping pong or assembling a Lego structure. In this context, there is continual feedback and response between the user and the system as the user performs the task. Our team was interested in these questions:

• Do WCA users change their behavior in response to system delays?
• Are their behavioral changes due to cognitive and/or emotional effects?
• Do they show any physiological responses to system delays?
• Do users’ personality characteristics impact their response?

They conducted a study with 40 undergraduate students interacting with a Lego assembly WCA application that was modified to introduce controlled system delays. The students’ responses were measured with application traces, biometric sensors and video captured from each session.

What they found was a clear indication of the pacing effect. When the system was slow giving responses, users slowed down their own execution of the task. When the system sped up, they sped up. However, it took some time for users to increase their pace after the system accelerated. While the team hypothesized that these changes would be, at least in part, due to frustration, anger or other emotions, they did not see any measurable indication of this in users’ heart rate, movement, EEG or galvanic skin response. They concluded that the pacing effect is primarily driven by a cognitive adjustment to the pace – just as I did in response to the slow updating of the Covid vaccine site.

They also looked into whether there was any correlation between user pacing response and user personality traits as measured by the Big Five Inventory of Personality and the Immersive Tendencies Questionnaire. They found that the traits of neuroticism, focus and involvement play a role explaining the differences between individual users. In this study, neuroticism, although not in the same sense as the common conception of that term, was the trait most correlated with user pacing. Here, it is more about the student’s personal tolerance for delay.

These results provide some insights for the designers of augmented reality applications – especially WCA applications:

• System slowdowns can lead to substantially longer user task completion times. This effect can cause greater than expected system resource consumption. Applying more resources to keep the pace may paradoxically result in use of fewer resources overall.
• And, given users’ slowed return to pace after the system recovers, it can be more valuable to use resources to keep the pace than to apply additional resources when system delays occur.
• On a multi-user system, a resource allocation bias toward fast users may be more efficient than a “fair” allocation of resources between fast and slow users. It allows the fast users to complete their tasks quickly and leave the system.
• It may be possible to maximize user satisfaction and resource efficiency by adjusting resource allocation in favor of those users known to be sensitive to system delays.
• Previous research has also shown that when delay occurs is also important. In WCA applications, delay matters most when the user has finished a step and is waiting for the next instruction.

As people use new application classes like augmented reality, our understanding of human-computer interactions expands to include new ways of interacting. And, because edge-native applications are very often highly interactive, we believe that the growth of the edge will be a fertile ground for new learnings. We welcome feedback and new ideas in this space – there is still much to discover. For more information, see the following references. By the way, I’m happy to say that my persistence paid off and I am now fully vaccinated!

FOR MORE INFORMATION

1. Olguín Mu?oz M, Klatzky R, Wang J, Pillai P, Satyanarayanan M, Gross J (2021) Impact of delayed response on wearable cognitive assistance. PLoS ONE 16(3): e0248690. https://doi.org/10.1371/journal.pone.0248690
2. Ha K, Chen Z, Hu W, Richter W, Pillai P, Satyanarayanan M. Towards Wearable Cognitive Assistance. In: Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services. MobiSys’14. New York, NY, USA: ACM; 2014. p. 68–81. Available from: https://doi.acm.org/10.1145/2594368.2594383
3. Junjue Wang, Ziqiang Feng, Shilpa George, Roger Iyengar, Padmanabhan Pillai, and Mahadev Satyanarayanan. 2019. Towards scalable edge-native applications. In Proceedings of the 4th ACM/IEEE Symposium on Edge Computing (SEC '19). Association for Computing Machinery, New York, NY, USA, 152–165. DOI: https://doi.org/10.1145/3318216.3363308
4. Jim Dabrowski, Ethan V. Munson, 40 years of searching for the best computer system response time, Interacting with Computers, Volume 23, Issue 5, September 2011, Pages 555–564, https://doi.org/10.1016/j.intcom.2011.05.008
5. Wearable Cognitive Assistance Blogs:

https://www.openedgecomputing.org/when-the-trainer-cant-be-there/

https://www.openedgecomputing.org/putting-it-all-together-with-cognitive-assistance/