What did you think would happen as a result of you logging that accessibility bug? Part 1

This article discusses the many ways in which an accessibility bug gets resolved, how it’s often not going to be resolved as “Fixed”, and provides some thoughts on how to avoid the accessibility bug being logged against your product in the first place.

Prologue

Of all the bugs I’ve investigated, there’s one that stands out as being the most fun. Amongst the repro steps was one that required me to play Sabotage by the Beastie Boys. And for the hours that flowed during the investigation, I found myself needing to run through the repro steps again and again, just in case I missed something.

I’ve never been so disappointed about having to resolve a bug, and to move onto the next.

Introduction

Q: What’s the most effective way to not ship bugs in software?

A: Don’t ship software.

That said, many solution builders do like to ship software. After all, software can help people do all sorts of great things. And overall, I’d say that most solution builders would prefer to ship a high-quality product for their customers, rather than a low-quality product. But despite their hard work to deliver that high quality experience, there’ll always be bugs.

So as those bugs get unearthed, ideally before the product ships, people will log them in some database and feel good that thanks to them, the product’s customer experience is one step closer to where it needs to be.

Or is it?

What happens in reality to all those logged bugs? Perhaps they all get fixed, because after all, it’s only software. Or is that goal more challenging than it seems?

This article considers the various ways an accessibility bug might get resolved, and provides thoughts on how those resolutions might over time skew more to a resolution of “Fixed”.

Q: What’s better than resolving a bug as “Fixed”?

A: Not having a bug in the first place.

So this article also provides some thoughts on how to avoid bugs existing at all.

What is a bug?

I’m sure there are lots of definitions for software bugs out there, and I’ll not add another one here. But one thing I am particularly interested in is the fact there’s a difference between (i) something that’s a flaw in a software product which negatively impacts the customer experience, and (ii) a tracking item in a database. The term “bug” can be used for both of those things, and those things are not the same.

Sometimes a tracking item is logged in a database, and it does not uniquely track a flaw in a product. As such, it’s unlikely to benefit the customer because while the product team is investigating such tracking items, they’re not doing something that’s more helpful for the customer.

Examples of resolutions of such items are "Not Repro" and "Duplicate".

Not Repro

There are many reasons why someone investigating a bug might not be able to reproduce the bug. For example, perhaps the bug was already fixed.

Tip: Anyone testing a product should use an official version of the product that’s approved for testing.

Tip: Anyone involved with testing and fixing the product should be aware of configurable settings in the tools being used for testing. A bug might only show up when a testing tool has a particular setting active.

To give an example of the point about a tool’s settings, I’ve seen multiple times product teams are not able to reproduce some particular bug relating to use of the Narrator screen reader. And that was because the person who logged the bug had a particular Narrator setting, (such as its Verbosity, or Context Verbosity, or “Hear advanced detail”,) different from how the person trying to reproduce the bug had the setting.

Duplicate

Ideally before any bug is logged, the person logging the bug would make a reasonable effort to check whether the bug’s already been logged. But given that it can take way longer to perform a solid verification of that than simply logging new bugs, duplicate bugs are inevitable. Besides, how can someone testing the app know whether two bugs on separate pages in a product are due to the same issue in a re-used common control, or are indeed two quite separate bugs.

Tip: Make a best judgement around whether a bug seems a duplicate, and if only one bug is logged, make sure that bug includes details of all the places where the customer experience seems affected. That bug can’t be fixed until all those places are addressed.

I wouldn’t say the challenge of dealing with time-consuming Not Repro and Duplicate bugs is particularly different for accessibility bugs relative to any other type of bugs, but it’s always worth considering how product teams can avoid spending time on things that won’t ultimately improve the customer experience.

The resolutions in the following section are I’d say much more interesting from an accessibility perspective.

Resolutions

The following bug resolutions are discussed in this section:

• By Design
• External
• Won't Fix
• Fixed

By Design

A “By Design” resolution means that everything is working as expected, and there is no flaw in the customer experience here. So if there is no flaw, why was the bug logged?

Below are a couple of the more interesting reasons.

My UI’s not right? Oh yeah? Says who?

Some people might feel it’s obvious what the expected behavior for UI is. After all, they’ve used UI for years, and so when they press tab at some control type and something happens which they don’t expect, then obviously it’s a bug. But I’d say things become more complicated if we start discussing a range of types of UI, such as UWP XAML, Web (on a variety of browsers,) Win32, WinForms, and WPF. Anyone care to hazard a guess on how all control types in all those cases react to all types of input?

And so if there’s no apparently-official rule about how all types of UI must behave, then someone might feel that their UI is not broken. I’ve yet to hear: “The Magna Carta doesn’t say my ComboBox must respect Alt+DownArrow”, but I expect it’s only a matter of time.

But where there are guidelines for UI, we can leverage them. There’s a great deal of help around keyboard interaction models for lots of web widgets at WAI-ARIA Authoring Practices 1.2 (w3c.github.io). Only the other day I was learning from Tabs how it’s optional for a newly focus tab to become active when it gets focus. So by default, it’s not a bug if a tab doesn’t become active on focus. Rather, if that behavior results in the best customer experience, that would be by design.

Also, there’s lots of documentation on Windows desktop controls for different types of UI, and in addition, there’s documentation on the expected UI Automation (UIA) representation for such controls at Supporting UI Automation Control Types. So if you’re thinking of customizing UI in a way that impacts its UIA ControlType property, you’ll want to make sure that what you end up with respects all that it should for that type of control. For example, if you end up with something that calls itself a ComboBox, it better support the UIA ExpandCollapse pattern, as detailed at ComboBox Required Control Patterns.

Tip: Bugs should not be logged against a non-web app simply because it behaves in some way that’s different from how web UI behaves. If the app is using a standard control, and its UI is behaving in the way that’s expected for that type of control, then there’s no bug in the app code.

I would add that I think it can be difficult to know what type of UI is presented by a product. Because I often must be sure of that before I comment on something, I’ll point the Accessibility Insights for Windows (AIWin) tool to UI, and check the UIA FrameworkId property. I talk in detail about that at UI Automation: An Incomplete Guide for UI builders – Part 1.

Tip: If you work with multiple types of UI, be sure you know exactly what type of UI it is before commenting on the expected behavior for that type of UI.

Tip: When resolving a bug as By Design relating to how some control behaves, point out that the control is consistent with all other controls of that type on the device. (If it isn’t consistent, then further investigation is justified.)

There’s nothing wrong with my programmatic interface

All products ship with a visual interface conveyed through pixels on the screen, and a programmatic interface conveyed through an API. The programmatic interface is as important to some of your customers, as the visual interface is to some of your customers. For UI running on Windows, the Accessibility Insights for Windows tool can be a very helpful tool for really getting to know the programmatic representation of a particular piece of UI. An assistive technology (AT) like the Narrator screen reader will access that programmatic interface, and make whatever subsequent announcements it feels are the most helpful to your customers.

But say your customer’s using Narrator’s forward navigation commands to move through all the controls in your UI, and instead Narrator moves backwards, is that a bug in your UI?

The more interesting question is really: Is your programmatic interface a great semantic match for your feature? If your programmatic interface is a great semantic match, then you’re serving your customers well. If it’s not a great semantic match, for example, the order of the UI elements in the programmatic interface is not a logical match for the feature, then your customers will need you to fix that.

Tip: You cannot deliver a usable (never mind great) experience for all your customers if you don’t know the programmatic interface of your product. On Windows, the Accessibility Insights for Windows (AIWin) tool can help you with that.

Figure 1 below shows the AIWin tool reporting the programmatic hierarchy of the Windows Run dialog, as exposed through the UI Automation (UIA) API. The controls on the dialog are exposed through UIA in the logical order.

Figure 1: The AIWin tool reporting the UIA hierarchy of the Windows Run dialog.

The Edge browser’s developer tools can also be helpful for examining the programmatic interface associated with some widget. For example, if the Narrator screen reader doesn’t announce a name with a button on a page, is that an issue with Narrator, or with the button? Figure 2 below shows the Edge browser’s developer tool presenting the accessibility interface for a button which only shows a left chevron on a page. The button has an accessible name of “Previous image”.

Figure 2: The Accessibility dev tools in Edge reporting that the programmatic name of a button that visually only shows a chevron, is “Previous image”.

Tip: Don’t log a bug against UI simply based on a screen reader’s announcements at the UI. Check the programmatic interface for the UI, and consider whether that interface is a good semantic match for the feature.

Important: Delivering a great experience for all customers takes time and effort, whether it’s for the visual interface or the programmatic interface. If an individual is building UI and is not prepared to put in that time and effort despite being supported by their management, then they’re in the wrong job. If on the other hand they want to put in that time and effort and they’re not being supported by their management, then they’re in the wrong organization.

External

An “External” resolution means there is a flaw in the customer experience when using the product, but the root cause of the flaw is in some component that’s not owned by the product team.

There are a variety of ways that an external bug can find its way into a product, some of which are described below.

Use of a component owned elsewhere in your organization

When your organization owns the external component, at least there can be a formal decision made at a high level as to how accessibility bugs in that component will be addressed. This can be more challenging if that component is now legacy, and is having no updates made to it, but still a formal decision can be made with full transparency, enabling all teams involved to move forward together in the best interests of all customers.

Use of a component not owned by your organization

In some cases, such a component is open source, and technically it may be possible to update the component directly to fix some accessibility issue. That path can be challenging in that many product teams will already have way more work that they’d like to complete than they have time for, so tasks around fixing some open-source product which they don’t own are not likely to fit nicely into their schedules. For other components which are not open source, the product team is wholly dependent on the manufacturer, and their abilities and interests around building accessible products.

Important! (This is way too important to call a “Tip”)

One of the most important things you can do to help all your customers is consider the accessibility of some dependency that you don’t own, before you take a dependency on it. If you end up with bugs which block customers from using your product, and those bugs are caused by inaccessible dependencies, you either (i) ship a product that’s not usable and not compliant, or (ii) remove the use of the dependency and delay your product from shipping.

So always consider the accessibility of a dependency before taking a dependency on it, and deliberately choose the dependency that’s most accessible. Don’t trust statements made by 3rd parties that their components are accessible. Rather do some research and test out demos that they provide, and where possible, learn what accessibility bugs are already logged against the dependency.

A classic example of where products ship inaccessible UI relates to charts and diagrams. Why build your own charting infrastructure if someone else’s charting library is already available and does all you need? Well, a charting library doesn’t do all you need if the results are inaccessible.

I’d say an interesting practical example here is the Highcharts pie chart demo. Below are some of the things I might consider as I interact with that pie chart.

The pie slices are filled with different patterns of different colors, and so are distinguishable by customers who leverage colors and by those who don’t. The slices are delineated by white space.

I can navigate through the slices with the arrow keys. Figure 3 below shows a magnified view with Windows Magnifier running at 400%. As I arrow to a pie slice, the magnified view changes to bring the slice gaining keyboard focus into view.

If I examine the pie chart with the Accessibility Insights for Windows (AIWin) tool, it reports that each slice of the pie is exposed as its own UIA element, whose UIA ControlType property is Image. The UIA Name property conveys what each slice relates to, and also the size of the slice as a percentage of the pie. Other UIA properties include IsKeyboardFocusable and HasKeyboardFocus, which reflects the fact that customers can move through the slices using the keyboard. Figure 4 below shows the AIWin tool reporting all this for a slice in the pie chart.

Figure 3: The magnified view of the screen, with the pie chart slice with keyboard focus in the view.

Figure 4: The AIWin tool reporting the UIA hierarchy for the slices in a pie chart, along with various UIA properties associated with the slices.

So when you consider adding a dependency on a charting library or any other library rendering UI, compare the accessibility of the various options available to you, and on behalf of all your customers, deliberately choose the option that’s most accessible.

So what if no option is accessible?

I considered whether I could embed HTML directly into this LinkedIn-hosted article, to demonstrate an accessible pie chart in action, but as far as I can tell, embedding HTML like that’s not supported here.

But this raises the question of what can you do when there’s no option that seems to generate the accessible chart-related UI that your customers need? In that case, a key point is that all important data conveyed in any way through the chart must be accessible to all customers.

So say I create a pie chart showing some (made up) resolution type percentages for accessibility bugs in some product. Figure 5 below is an image of that chart. I don’t want to set a really long accessible name on the image, with that name containing all the data of interest, as that’s unwieldly. Rather I’ll set a name on the image that describes the purpose of the image, and provide an accompanying data table which neatly presents the data as text. I’ll also state in text before the image that the data conveyed in the image is contained in a table following the image. By doing that, all the data is accessible to all my customers. So while I’d like to be able to include an accessible chart here, the absolute minimum requirement is that all data must be easily accessible to all customers.

Note that the data table can help to address a number of associated issues with the chart shown in the following image, including the fact that color alone is being used to fill the pie slices, and the text shown on some slices is very low contrast against the text background.

Note also that not all the resolution types in the chart below are wholly serious.

Figure 5: A pie chart titled “Bug Resolution by percentage” for seven resolution types.

And after saying all that, it turns out that I don’t know how to add a data table to this LinkedIn article either. That wouldn’t be a problem in your web product. So for now, I’ll simply list below the data conveyed visually in the image.

Bug Resolutions by percentage

Fixed: 60%

External: 20%

By Design: 10%

Won't Fix: 4%

Duplicate: 3%

Not Repro: 2%

Sabotage by Agents of Galaxy 5: 1%

One final point on External bugs relates to something I mentioned earlier when discussing By Design bugs, and that’s the importance of understanding your UI’s programmatic representation. Say a bug relating to the screen reader experience at a web product’s UI, gets logged against the web product. The web product team might immediately declare the bug to be due to the screen reader, and resolve the bug as External. The screen reader team might then at some later time feel there’s no evidence at that point which indicates this is a bug in the screen reader, so the bug heads back to the product team, and the product team immediately resolves it as External to some browser, in the hope the bug will just go away. And so it goes on.

Before multiple bugs are created and bounce around wasting many people’s time, at the expense of planned work that really needs to get done to help customers, a product team needs to understand the programmatic representation of their UI.

And I do acknowledge that I’m saying this from the viewpoint of someone who’s lucky enough to have a tool like Accessibility Insights for Windows to examine the UIA representation of UI running on Windows. In some other scenarios, accessibility-related tools may be less powerful.

Won’t Fix

A “Won’t Fix” resolution means it’s agreed that there’s a flaw in the product, and it’s in code owned by the product team, but the team is not going to fix it.

With any product, as teams find their ship date getting nearer and nearer, there’ll be a long list of active bugs to consider. Some of those bugs will relate to accessibility, and many will not, and it won’t be practical for the product team to get close to fixing them all. That’s the reality of the world we live in. So there’s careful consideration of the bugs, and it’s decided which bugs have such low impact on the customer experience, that customers won’t be significantly impacted. Rather, even with some bug in the shipping product, customers can reasonably access all the data and functionality in the product, and can reasonably complete their tasks. As such, the bug may get resolved as Won’t Fix.

But who’s to say what’s reasonable?

This is where we need to be really careful as to who’s making that decision. Is it a reasonable workaround for a bug if a customer has to do one of the following:

1. Guess where keyboard focus is because there are no visuals on the screen making that clear?
2. Require the use of mouse or touch once while completing a form? C’mon, it’s only once.
3. Use a screen reader’s backward navigation commands, in order to move through controls that are visually shown in a forward direction?
4. Complete a selection at popup UI in 5 seconds.
5. Ask someone else for help in using the product.

Sometimes there’s a rush to declare a product as being compliant, due to the product having to ship within a few days. Many years ago the person across the corridor for me had a sign on his door: “A lack of planning on your part does not constitute an emergency on my part”. And while I actually thought that was pretty unfriendly, it does make a point. Product teams can’t leave accessibility to the last minute, and then think it’s ok to declare that many of their accessibility bugs are low impact because they don’t have time to fix them before they ship.

Tip: Plan and work on accessibility throughout the product cycle, and avoid the frantic scramble to try to become compliant just before you ship.

Tip: The people considering the level of impact of an accessibility bug, must have knowledge in the related area of accessibility. Don’t expect someone who consumes the default visuals of the product, and happily uses both keyboard and mouse, and has no experience with accessibility, to magically be able to appreciate all the practical nuances of some workaround.

Tip: By default, just fix the accessibility bug. Sometimes it can be quicker to do that than have multiple meetings with multiple people discussing whether the bug needs to be fixed.

Fixed

A “Fixed” resolution means it’s agreed that there’s a flaw in the product, and it’s in code owned by the product team, and the team’s fixed it.

Fixing a bug is a good thing, but it’s worth considering how efficient the process of fixing the bug was, and also very importantly, what can be done to avoid the bug in the future?

If a product’s being designed without consideration for accessibility, it’ll exhibit many types of accessibility bugs. Typically the three largest groups of bugs will relate to keyboard accessibility, use of color, and programmatic accessibility.

Important! (This also is too important to be considered a tip)

The only way a product stands any chance of having a smooth path to being accessible is for careful thought around its accessibility being applied during the design phase.

There’s no technical reason why the majority of accessibility bugs can’t be avoided these days. By designing a product’s keyboard experience, and by designing how use of colors can result in a great experience for all customers, and by designing a great programmatic interface, there won’t be the hundreds of accessibility bugs that traditionally exist in a product.

I believe that once the software industry really starts designing for everyone, the proportion of keyboard accessibility and color-related bugs will plummet, because most solution builders can quickly understand the rationale, goals, and UI details in those areas. Conversely the proportion (not the absolute number) of programmatic accessibility bugs will rise, because it will take longer for the majority of solution builders to get a grip on that.

To illustrate that point, conversations go something like the following:

Q: Did you make sure that no important information is being conveyed through color alone?

A: Yeah. We found one case where the fill color in an element changed from green to red to indicate there was a failure, but we fixed that.

Q: And did you know your ComboBox-like control isn’t raising a UIA_ExpandCollapseExpandCollapseStatePropertyId PropertyChangedEvent when its dropdown appears?

A: Are you talking to me?

Education is key here. If everyone involved with building products learns more about what it’ll take to support all customers, things can really change. Of course, with any group of people large enough, there’ll be people who don’t care about anyone but themselves, but I like to think they’re not the majority. Most people I know want to build high quality products for everyone; they just need support in both training and tooling to make that practical.

Some technical and not-technical resources you may be interested in are listed below.

• Accessibility at a glance
• Accessibility fundamentals
• Accessibility 101
• WAI-ARIA Authoring Practices 1.2
• Sarah Higley’s articles
• Marcy Sutton’s Start Building Accessible Web Applications Today
• Marcy Sutton’s Accessibility in JavaScript Applications
• Accessibility in Windows 10
• Introduction to UIA - At a Glance
• UI Automation: An Incomplete Guide for UI builders – Part 1

While in the long term it should be practical to avoid the majority of accessibility bugs that get logged today, for quite some time there will continue to be many bugs, and so the efficient resolution of those bugs is critical. This is where tools like Accessibility Insights can help, in that not only can it help draw your attention to potential accessibility bugs, it can also give you a head start on fixing them.

For example, Figure 6 below shows the Accessibility Insights for Web (AIWeb) tool describing how the contrast of text against its background on a web page, does not meet a web standard. In this case the AIWeb tool cannot provide specific RGB values to fix the bug, as it can’t know whether its most appropriate for the text color or the background color, (or both,) to change. But it does make it clear which colors are problematic, and what the goal is.

Figure 6: The Accessibility Insights for Web tool reporting a color contrast failure on a web page, and providing details about the failure.

Figure 7 below shows the Accessibility Insights for Windows (AIWin) tool reporting that a list in an app’s UI has no programmatically accessible name. In this particular case, a “View code sample” link provided by AIWin references documentation containing a code snippet showing how to add an accessible name to a list, and which is appropriate for the type of UI in the app.

Figure 7: The Accessibility Insights for Windows tool reporting a list name failure against desktop UI, and offering a link to related dev help. In the background is that dev help open in a browser.

Summary

I called this article a “Part 1”, because discussions around causes of bugs, and the paths those bugs can take, are way bigger than what I’ve included here. But this article discusses some of the more common causes and paths, and I wanted to make the point that simply because an accessibility bug is logged, does not mean a product’s going to be updated as a result.

Ideally, bugs would be avoided through education, and where there are still bugs, there’d be an efficient path to fixing them. Having to spend time on bugs that legitimately get resolved as By Design is not good for your team or your customers, because you’ve been pulled away from doing work that could help your customers more. And bugs that ultimately get resolved as External, mean your customers have to handle the consequences of the fact that you took a dependency on an inaccessible component.

So:

• Don’t take a dependency on an inaccessible component that you don’t own. If you feel you have no choice but to do that, support all your customers by taking additional action in the UI that you do own.
• Understand the programmatic interface of a product before logging a programmatic accessibility bug against it. Don’t log a bug against UI based only on a screen reader’s announcements at the UI.
• Understand the programmatic interface of your product before declaring that a programmatic accessibility bug isn’t caused by your product.
• If you’re a Designer, take the time to learn how to design an accessible product. If you’re a Dev, take the time to learn how to implement an accessible product.

And given that it’s December, here’s one more resolution worth considering...

My New Year’s Resolution: Ship products that deliver great experiences for all my customers.

All the best for 2021 everyone.

Guy