How can you evaluate AR using tools and frameworks?

1 AR Evaluation Goals

Before you start evaluating your AR application, it's important to define your evaluation goals. Ask yourself questions such as: what are you trying to achieve with your AR project? Who are the target users and what are their needs and preferences? How do you measure the success of your AR application? Depending on these goals, you may want to focus on different aspects of your AR application. For instance, consider its functionality - does it work as intended? Does it have any bugs or errors? Does it perform well on different devices and environments? Additionally, consider its usability - is it easy to use and understand? Does it provide clear feedback and guidance? Does it match the user's expectations and mental models? Furthermore, reflect on its user experience - is it engaging and enjoyable? Does it create a sense of presence and immersion? Does it elicit positive emotions and attitudes? Finally, evaluate its impact - does it have any positive or negative effects on the user's behavior, learning, or performance? Does it solve a real problem or add value to the user's context?

2 AR Evaluation Methods

Once you have defined your evaluation goals, you need to choose the appropriate methods to collect and analyze data from your AR application. These methods can be broadly categorized into two types: quantitative and qualitative. Quantitative methods use numerical data and statistics to measure and compare aspects of your AR application, while qualitative methods use non-numerical data and insights to explore and understand aspects of your AR application. Examples of quantitative methods include surveys, questionnaires, ratings, scales, tests, scores, logs, analytics, eye-tracking, biometrics, etc. Qualitative methods include interviews, focus groups, open-ended questions, think-aloud protocols, diaries, video recordings, audio recordings, annotations, etc. Utilizing a mix of both quantitative and qualitative methods can help you gain a comprehensive understanding of your AR application.

3 AR Evaluation Tools

When applying evaluation methods to your AR application, you may need different tools depending on the type of data and method used. For instance, general-purpose tools like Google Forms, SurveyMonkey, Qualtrics, Excel, SPSS, R, and Python can be used for various types of data and methods. For more specific tasks, you can use tools like ARToolKit, Vuforia, Unity Analytics, ARCore, ARKit, ARFoundation, AR Debugger, and ARCore Elements.

4 AR Evaluation Frameworks

Organizing and guiding your evaluation process can be made easier by utilizing frameworks. These models or structures provide a systematic and consistent way of conducting evaluation, assisting you with defining goals, selecting methods, using tools, and communicating results. Some examples of frameworks that can be used for AR evaluation include ISO 9241, TAM, SUS, MEC, and PRESENCE. ISO 9241 is a standard that defines the principles and requirements for the usability and user experience of interactive systems. TAM is a model that explains the factors that influence the user's acceptance and adoption of a technology. SUS is a scale that measures the user's subjective satisfaction with the usability of a system. MEC is a model that defines the dimensions and components of user experience. PRESENCE is a concept that describes the user's sense of being in a virtual environment.

5 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?