What are the best metrics to measure the accessibility of your product for users with disabilities?

Product manager’s job is to make your product inclusive and accessible for all. WCAG is a standard that helps to make sure your website or application meets accessibility standards and the product can be used by persons with special needs. There are 4 major priciples of WCAG standard perceivable, operable, understandable and robust. During the development as a PM you need to make sure your cross functional team keeps WCAG as a pre-requisite part of frontend development and testing by including necessary tags, annotations, color contrast, etc Your team can leverage magnifier, screen reader, high contrast mode, keyboard tabs, W3C site and other premium tools to check for WCAG complaince. Some tools and add-ons can generate reports too.

The WCAG serves as the gold standard guidelines, continuously evolving to incorporate recent technologies and expert feedback. From an Indian legal perspective, companies are required to comply with two specific guidelines in accordance with their commitment to the Rights of Persons with Disabilities (RPWD) Act 2016: 1. Bureau of Indian Standards (BIS) IS 17802 Part 1 and 2 2. Guidelines for Indian Government Websites (GIGW)

Product accessibility for disabled users is ensured by adherence to WCAG standards. Screen readers are used for testing, keyboard navigability is confirmed, and high color contrast is maintained. Alternative text for visuals is provided. Testing is performed with individuals with disabilities, clear error identification is guaranteed, and content readability is ensured. Compatibility with assistive technologies is checked, captions for media are provided, and customization options are offered. Focus order is kept logical, and support channels are made accessible. Automated tools are utilized for compliance tracking, and updates on legal accessibility requirements are followed.

Apart from tracking number of compliance issues identified, time required to remedify those issues is a critical metric too for accessibility. Some fixes could be quick while others might require weeks due to shared resources and other needs. The compliance authorities usually specify SLA for accessibility related compliance resolution.

This depends on the type of product and the region it is being used, for example, if its - web-based product then WCAG 3.0/ARIA can used - video conferencing tool along with WCAG 2.1/ UDL metrics can be applied applied Also these metrics to be updated as per sector and region the product is deployed. For example: - If it is a video conferencing app that is used in the education sector then it must comply with CVAA ( 21st Century Video accessibility act), ADA (Americans with Disabilities ) rules, and WCAG to support features like subtitles, closed captions, audio descriptions support Hence it is better to get your products get accessibility certified depending on region, sector, target users

ADA and WCAG tools provides you guidance to adhere with those standards but as a Product manager it is not enough. You need to work backwards with your users with special needs to understand what are their pain points with your product and what are their needs to meet their unmet needs when it comes to making your product accessible and inclusive for all. You can collect user feedback through surveys, interviews, closed user group sessions or performing usability testing. You can use various tools to collect user feedback like Qualtrics, Pendo, UserZoom, etc to segment and later filter relevant details related to accessibility topics for your product.

Besides conducting usability tests with the intended user segment, a fantastic practice I've personally found incredibly valuable is to engage coworkers and colleagues with accessibility needs. Their firsthand feedback can be quick, free, and iterative, offering unique insights ??

Most of the products are designed by normal users, and hence the "needs" of special users are overlooked. So, user feedback becomes non-negotiable when it comes to accessibility. Traditional feedback mechanisms like forms and surveys will not be sufficient to understand the nuances. Recommended to perform offline interactions that offer a firsthand view of the obstacles encountered by them for simple navigation. Accessibility is an iterative process where feedback is used to improve the product, and if possible, special users should be part of the team before signing off on a new build. It is important to recognize that mere conformance with WCAG doesn't ensure a seamless user experience, which is achieved through user feedback.

Absolutely! While qualitative feedback paints a vivid picture, don't overlook the power of product analytics. They unveil the hidden patterns—what's in the spotlight, what's in the shadows. You can also consider crafting a heatmap; it's like turning data into a colorful masterpiece! ??

As a Product manager you can leverage quantitative data from users with special needs for your product. It can provide you guidance on their usage pattern, where they are getting stuck, if there is high bounce rate, inability to perform task success, trends and pattern to identify user funnel and what to optimize to help them complete their goals with your product. You can use tools to collect and analyze quantitative data using analytics tools like Google Analytics, Hotjar, Tableau, Adobe, Mixpanel, etc. But most of these tools collect user anonymized data so make sure you organize usability testing for these user group to analyze their data in a dedicated environment and make informed decisions using product metrics.

Data (metrics) is a powerful tool when wielded effectively. Metrics can serve as evidence of the benefits derived from product enhancements. To validate hypotheses, product managers must possess a thorough understanding of the accessibility user cohort. Furthermore, data can aid in issue identification and detection of anomalies within accessibility user patterns, thereby highlighting any shortcomings in the build.

Once you have made improvements on your product to adhere with WCAG standards by collecting enough quantitative and qualitative data, you can start combining all those and convert it into an Accessibility score for your product. This single number will be your guiding compass as a Product manager to continue improve and enhance your product to meet the needs of your user group with special needs. You can set a baseline before the start of this exercise and also get details about your competitive product on their score using your research team. Once set, continue to improve this score for your users. You can leverage tools like WAVE, AccessibilityOz, independent WCAG auditors to track this score for your product.

This is more of a business side but as a Product manager you can measure the ROI of making accessibility changes to your product. What is the cost of this complete exercise, what improvements you see with your product after making it WCAG compliant in terms of growth, retention of users and how it converted into monetization for your product. Make sure it ties to your product mission, vision, strategy and leaders are on the same page while taking accessibility initiatives as it is time and cost intensive. If the product is more user focused and wants to solve their problems and meet their needs, making accessibility improvements should be a no brainer and it should be automatically part of your strategy!!!

As a PM you do not need to measure Accessibility maturity but create a culture within product teams and your organization to make sure any enhancements, changes or new integration or adding a new product or feature is developed by keeping accessibility standards at forefront of development, testing and received necessary user feedback.

Accessibility initiatives should be driven from the company level, as relying solely on a product manager's efforts will not be sufficient. It begins with sensitizing all stakeholders to the importance of accessibility and fostering inclusive design that resonates with all users. Accessibility should not be treated as a one-time task but rather integrated into the product development process and roadmap. To support this approach, companies should establish processes and checklists that facilitate a seamless product cycle while prioritizing accessibility considerations.

As a product manager, effectively measuring product accessibility for users with disabilities involves a comprehensive approach. Key metrics include compliance with standards like WCAG, gathering direct user feedback, and analyzing usage data to understand behavior patterns. Additionally, calculating an overall accessibility score helps benchmark against industry standards. Assessing the ROI of accessibility initiatives is crucial for aligning with business goals. Lastly, evaluating the maturity of accessibility integration in the product development process is essential. These metrics collectively guide improvements, ensuring the product is inclusive and meets diverse user needs.

Sometimes early stage products don't bother about accessibility in design, but this becomes a huge tech-debt issue when the product suddenly starts scaling. Many customers insist on a level of accessibility scores/maturity on RFPs. For my products, I always insisted on following steps - Accessibility in all UX designs (mobile, web, devices) - Design QA should include in house accessibility experts - Automated testing should always include accessibility score checks - User testing should include disabled users in sample regions targeted for deployment - Accessibility legal experts should get involved in product documentation and EULA - A pre-launch formalities should include acessiblity certifications as a must

Performance metrics would be useful to assess for measuring the accuracy, speed and reliability of the tools. Assistive technologies rendering and interpreting content, screen readers reading text and voice recognition software processing spoken commands.