The Lost Art of Tutorial Levels: A Reflection on User Acceptance Testing
Ashish Tripathi
Strategic Associate Product Executive | Driving Innovation & Growth in Marketing, E-commerce, and Energy Commodities | Expert in Product-Market Fit & Cross-Functional Leadership
Recently, I watched "Semi-Ramblomatic: The Lost Art of Tutorial Level by Ben “Yathzee” Croshaw. The video explores the evolution of tutorial levels in video games, which were once common before the trend shifted towards integrating tutorials into the initial gameplay experience. This transition occurred shortly after games started having more memory space but still came with physical game/user manuals. One notable example discussed in the video essay is "Half-Life."
Despite the shift, tutorial levels still exist in modern games like "Prince of Persia: The Lost Crown" and "Vader Immortal I." The tutorial level persists even in today's gaming landscape.
I found the video fascinating due to the similarities between User Acceptance Testing
Advantages of User Acceptance Testing
Early Buzz
One advantage of early UAT is the opportunity to market your product and generate early buzz among your existing customers. By involving them in the UAT process, you can showcase upcoming features and provide them with special treatment as UAT users. These users get an advanced look at product updates and can request specific changes they’d like to see. This approach is particularly beneficial for high-profile B2B customers, allowing them to feel valued and heard.
Identifying Critical Issues
UAT environments are crucial for identifying critical issues that could prevent users from successfully using the product. By having users go through key customer journeys, you can ensure that the application functions as intended and catch user behaviors that might not be apparent in other testing phases. This process allows you to address any friction points within the application before it is released to the wider audience.
Testing Deployment, Hotfixes, and Updates
During UAT, you can also test your deployment structure
Disadvantages of User Acceptance Testing
Users Forget the Instructions When They Need Them
Studies show that 70% of users forget instructions from their initial training within 24 hours. Despite spending hours creating and deploying training for UAT, users often forget this training when using the product. I've had numerous calls where, 15 minutes before the closing window of a trade, users ask how to submit a trade. No matter how well we design the flow, if users are stressed and working on a new application, they will inevitably forget the instructions they received months ago during UAT.
Mitigation
To mitigate this issue, provide in-application and context-based training and user guides. Similar to how video games have evolved to offer in-game tutorials, business applications should offer accessible training within the application. This ensures that users can find help at any time and receive guidance while performing tasks. In video games, this is often a tutorial level that can be referenced or paused to review moves or combinations needed to progress. This approach can significantly reduce the likelihood of users forgetting their training.
领英推荐
You Cannot Trust the UAT Feedback
People are busy and often do not engage with UAT as if they are truly working on the product. The same issue exists with beta testing. Unless compensated, most users will not dedicate significant time to UAT tasks, which are typically outside their regular work responsibilities. As a result, feedback may be unreliable and not reflective of real-world use.
Mitigation
Ensure you have robust analytics and a standardized process for collecting feedback
Wasted Time for Unique UAT Setup
Setting up a separate UAT environment can be time-consuming and inefficient, especially when it involves creating test data and accounts. In my experience, creating large environments and test accounts to simulate a “real” setup often results in wasted time.
Mitigation
Consider creating pilot environments in production. A canary deployment strategy
Conclusion
The parallels between tutorial levels in games and User Acceptance Testing (UAT) are striking. Both offer advantages such as early buzz, critical issue identification, and testing of deployment processes. However, they also share disadvantages like users forgetting instructions and the unreliability of UAT feedback.
Just as tutorial levels have evolved from separate stages to in-game contextual training, UAT can benefit from adopting similar strategies. Incorporating in-application training
Moreover, creating pilot environments in production, similar to the canary setup used in game releases, can provide a more realistic UAT experience without the need for separate environments. By referencing the game tutorial concept, we can innovate UAT practices and improve the overall user experience.
By learning from the evolution of tutorial levels in video games, we can make UAT more effective, efficient, and user-friendly. Embracing these strategies will not only improve the testing process but ensure a smoother transition to full-scale product deployment.