A Review of Mobile Ap UX Questionnaires

The mobile app industry is very lucrative, having generated $45 billion in the U.S. and $253 billion worldwide in 2023. A good user experience is critical to the success of websites and mobile apps.

We’ve shown how perceived ease of use and perceived usefulness impact people’s intention to use and recommend software products. We’ve also shown how those intentions predict actual behaviors, demonstrating a chain of influence from UX metrics to important business outcomes.

Mobile apps are a distinct type of product that warrant their own measurement, but work on standardized mobile app questionnaires has been limited.

In this article, we review how mobile UX has historically been measured and then review three mobile-specific questionnaires: the MPUQ, mod-AUG scales, and SUPR-Qm?.

Read the full article on MeasuringU's Blog.

Summary and Discussion

The mobile app industry is large, and there is significant competition for users. Despite their limitations in diagnosing specific usability problems, standardized questionnaires are important tools in the overall assessment of the UX of mobile apps. Only a few standardized UX questionnaires have been developed specifically for mobile apps—the MPUQ, the mod-AUG scales, and SUPR-Qm.

Questionnaires developed with CTT and IRT play different roles in UX evaluation. CTT tends to generate sets of items that are optimized around the average level of multiple constructs. IRT, by contrast, optimizes around a questionnaire that reliably measures a fuller range of a single construct, from low to high, not just around the average. Whether CTT or IRT is the better approach for developing a standardized UX questionnaire depends on the measurement goals.

Use the SUPR-Qm to get a quick measure of the overall UX of mobile apps. When the measurement goal is a unidimensional measure that is sensitive to a broad range of knowledge or experiences (as is the SUPR-Qm), the more appropriate questionnaire development method is IRT (including Rasch analysis).

If there is a need to measure additional constructs, use one of the longer multifactor questionnaires. When the goal is the measurement of multiple constructs (as in MPUQ and mod-AUG scales), the more appropriate questionnaire development method is CTT.

There are practical issues with the current multifactor questionnaires. The multifactor standardized questionnaires are good from a purely statistical perspective, but with 72 items for the MPUQ and 78 items for the mod-AUG scales, they are not practical for rapid assessment of the UX of mobile apps, plus there are no published norms for interpreting the resulting scores (overall and subscale). This is, in part, due to using the methods of CTT to develop the questionnaires, which is good at identifying multiple factors but requires at least two items per subscale (a few more is usually better) and a lot of research to develop and maintain interpretive norms for questionnaire scores. Furthermore, the more specific the item content, the more likely it is to become less relevant over time (e.g., from the MPUQ, “Are the HOME and MENU buttons sufficiently easy to locate for all operations?”).

The SUPR-Qm is a quick measure of overall mobile app UX, but it could be more efficient. Examination of the Wright map in Figure 1 shows opportunities to streamline the SUPR-Qm by removing redundant items. Redundant items are those that are located around the same place on the y-axis (having similar logit positions). For example, items close to the exact center of the scale (e.g., AllEverWant and Delightful) have essentially the same measurement properties, so it doesn’t matter which one is selected for inclusion in a streamlined version of the SUPR-Qm. A key research question is how many items could be excluded and still have a streamlined version that produces scores comparable to the original version.

Future research: In future articles, we will discuss research we’ve conducted to replicate the original SUPR-Qm findings, streamline the SUPR-Qm, verify the stability of full and streamlined versions of the SUPR-Qm, and develop norms for the interpretation of SUPR-Qm scores.

For more details about this research, see the paper we published in the Journal of User Experience (Lewis & Sauro,?2025).

Read the full article on MeasuringU's Blog.

Upcoming Webinar:

20 Years of MeasuringU - Reflecting and affecting UX research for two decades at MeasuringU

Date: Wednesday, April 2, 2025

Time: 1:00 PM ET?| 10:00 AM PT

Hosts: Jeff Sauro PhD & Jim Lewis PhD

Sign up for our free webinar!

Who cares about what happened 10, 15 or 20 years ago?

While technology changes fast some of the most important questions in UX research are more enduring. Preparing for the future means understanding the past.

In this webinar we’ll reflect on key moments and topics that shaped the industry and how MeasuringU came to play a major role in the transformation of UX measurement.

In particular we’ll cover timeless topics including Metrics, Methods and Tools, focusing on:

• How do you measure UX?
• How has that evolved?
• How have UX methods changed?
• What sample size do you need?
• Why is this still a controversy after over 20 years!?
• How should you analyze data?
• What role will AI play?
• How bad is the current UX job market compared to the cycles seen in the last 20 years?

Join Jeff and Jim as they provide their perspective on the evolution of UX research measurement and methods since the genesis of MeasuringU.