What are the best ways to test CMS accessibility and usability with real users?

1 Define your goals and metrics

Before you start testing your CMS with real users, you need to have a clear idea of what you want to achieve and how you will measure it. For example, you may want to test how easy it is for users to create, edit, and publish content, or how well your CMS supports different devices, browsers, and assistive technologies. You also need to define the metrics that will indicate the success or failure of your tests, such as completion rate, time on task, error rate, satisfaction score, or accessibility score.

2 Choose your testing methods and tools

When evaluating your CMS with real users, you can choose from different testing methods and tools depending on your goals and resources. For example, usability testing involves observing and interviewing users as they perform tasks on your CMS. You can use tools like Zoom, Google Meet, or UserTesting to conduct remote usability testing sessions and record the user's screen, voice, and facial expressions. Accessibility testing checks how your CMS meets accessibility standards and works with assistive technologies. Tools like WAVE, AChecker, or Lighthouse can be used for automated accessibility testing, while tools like NVDA, VoiceOver, or JAWS can be used for manual accessibility testing with disabled users. A/B testing compares two or more versions of your CMS features or designs to measure how they affect user behavior and outcomes. Google Optimize, Optimizely, or VWO are tools that can be used to create and run A/B tests on your CMS and analyze the results.

3 Recruit and segment your users

Testing your CMS with real users requires recruiting and segmenting them according to your target audience and testing objectives. You can reach out to existing or past users of your CMS via email, social media, or in-app messages and offer them incentives like gift cards, discounts, or free access. Additionally, platforms like UserTesting, User Interviews, or Respondent can help you find and attract potential users who match your target audience. Furthermore, user segments can be created based on characteristics, preferences, or behaviors to tailor tests to different user scenarios and expectations.

4 Prepare and conduct your tests

Once you have your goals, methods, tools, and users ready, you can prepare and conduct your tests with real users. To ensure success, it's important to create a test plan that outlines the purpose, scope, and details of the tests. This should include the test objectives, tasks, questions, scenarios, metrics, and criteria. It's also wise to conduct a pilot test with a small sample of users to identify any issues or errors in your test plan or tools before conducting the full-scale tests. After refining the pilot test, you can then proceed with the full-scale tests with your selected users. Make sure to follow the test plan and script and record and document the data and feedback. Lastly, don't forget to thank your users and deliver any incentives or rewards.

5 Analyze and apply your results

The final step of testing your CMS with real users is to analyze and apply your results. You should use tools like Excel, Google Sheets, or Tableau to organize, visualize, and analyze the quantitative data from your tests. Additionally, you should use tools like NVivo, Dedoose, or Thematic to code, categorize, and analyze the qualitative data from your tests. With this data you can identify the strengths and weaknesses of your CMS and prioritize issues or problems that need to be addressed. Finally, you should apply the insights and recommendations to enhance your CMS and communicate and share your results and actions with your team, stakeholders, and users. This process is essential for making sure that your CMS is accessible and usable for your target audience.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?