Using Visual Studio’s integrated Accessibility Checker with a new .NET MAUI puzzle game
Guy Barker
Exploring how to make apps more accessible to more people, and sharing all I learn.
This article describes my experiences using Visual Studio’s new integrated Accessibility Checker with a demo .NET 7 MAUI puzzle game called Leaf Sweeper.
?
Links related to the content of this article include:
Source code for the Leaf Sweeper app, (specifically, any file with “Sweeper” in its name).
?
?
Introduction
I recently downloaded a Minesweeper puzzle game from the Microsoft Store, and as I tend to, the first thing I did was point the very handy Accessibility Insights for Windows at it in order to learn how the game presents itself to screen readers. The result was that the game is completely imperceivable to screen readers like Windows Narrator.
The Minesweeper game makes a screen reader aware that it presents a window called “Microsoft Minesweeper”, yet supplies no information at all about what’s inside the window.
Note: Programmatic accessibility on Windows is powered by the UI Automation API, (UIA). For an introduction into UIA, please visit UI Automation: An Incomplete Guide for UI builders.
Figure 1: The Accessibility Insights for Windows tool reporting the UI Automation hierarchy of the Microsoft Minesweeper app. The main UI for the game is represented by a UIA element whose Name is "Microsoft Minesweeper", and LocalizedControlType is "window". That element has no child elements.
So this got me thinking, if I were to build a similar game, how might I make it more accessible to more people? Of course I found the idea irresistible, and as I was building the game I heard the very exciting news that the latest update to Visual Studio 2022 Preview includes an integrated Accessibility Checker.
This article shares some thoughts on the Visual Studio Accessibility Checker, along with my new Minesweeper-inspired (but more accessible) Leaf Sweeper game.
For a more incremental progression on my thoughts while building the game, check out my recent tweets (@gbarkerz) starting from the moment I discovered the inaccessible Minesweeper game.
Figure 2: The Accessibility Insights for Windows tool reporting the UI Automation hierarchy for the grid of squares in an in-progress game of Leaf Sweeper. The grid is represented through UIA as a list whose name is "Sweeper Squares". The list contains sixteen list items whose names are one of the following: the count of nearby frogs followed by the index of the square, such as "1 nearby frog for stone 3", or "Query Frog for stone" followed by the index of the square, or "Leaf on stone" followed by the index of the square.
?
So just how useful is an accessibility checker?
Well, personally, I’d say you can’t ship an accessible app on Windows without an accessibility checker. But there are a few things I’d should add to that.
One of the biggest challenges around using an accessibility checker to improve the accessibility of an in-development app, is that if the checker isn’t used, then it can’t help. That might sound a little flippant, but it’s a very serious comment indeed. There are multiple reasons why in practice a dev might not run an accessibility checker, and for Visual Studio to now provide quick access to a checker directly from the IDE being used to build the app, is in my opinion an extremely important step forward. You might feel I’m exaggerating, but I spent years trying to get devs to manually run standalone accessibility checkers that weren’t integrated into their development environment, and often it was an uphill battle to say the least.
Another very important thing to remember when using an accessibility checker is that if a checker doesn’t find any issues with your app, that doesn’t mean the app is accessible. For example, the Accessibility Insights for Windows tool didn’t find any failures with the Minesweeper game, and that’s because Accessibility Insights didn’t know the game was there. Visual Studio’s integrated Accessibility Checker is a very helpful and quick way to check whether something seems problematic with your UI, but your customers still need you to periodically examine your UI more closely in order for you to feel confident that you’re delivering an accessible app.
Finally, I’d say that even if no issues are reported with Visual Studio’s integrated Accessibility Checker or Accessibility Insights for Windows, that still doesn’t mean your app is usable. Your goal is not to build an app for which no accessibility checker issues are found, rather your goal is to build an app which is very helpful to all your customers. So in addition to running the Accessibility Checker, include feedback from many diverse people during the design, development, and testing stages of your product life cycle to feel confident that you’ve done everything you can to help all your customers.
?
To get set up with Visual Studio’s Accessibility Checker, follow the steps described at Accessibility Checker announcement.
?
A few thoughts on the new .NET MAUI Leaf Sweeper game
I built the Leaf Sweeper game to help me consider some accessibility-related topics and to share related thoughts, and at the moment the game is fairly basic. It certainly has some bugs in its design and implementation, but I achieved what I set out to do, and made the code public for anyone wanting to review the steps I took relating to accessibility. And I have to say, I find it quite a relaxing way to pass the time, so perhaps it’s not a bad game in its own right.
As I was building the app, I did hit some apparent constraints in MAUI such as .NET 7 MAUI CollectionView doesn't support the UIA Grid pattern and .NET 7 MAUI CollectionView item's context menu is not invokable via a Narrator screen reader touch gesture, but I’d say the existence of these sorts of issues isn’t too surprising given how recent .NET 7 MAUI’s release was. And it’s really handy to be able to provide feedback directly to the MAUI team through MAUI Issues.
For the record, to workaround the lack of support for Narrator’s context menu touch gesture, I added a button in each CollectionView item which has no visuals and is not keyboard focusable. A screen reader can navigate to the button and invoke it to trigger the same action as that associated with the inaccessible context menu.
Before mentioning any specific implementation details here, as always, we should carefully first consider what level of accessibility can be achieved simply through good design and use of semantic MAUI controls. Please do read the Design and Semantic Controls sections in Considerations Around Building an Accessible Multi-platform App for a few related thoughts.
These were some of the things I considered early:
Figure 3: A lost game of Leaf Sweeper shown using the app's dark theme colours. There are twelve dark grey squares, and nine of those show a white number from 1 to 3. Four squares have a dark red background with light green symbols of a frog and a frowny face. One of those squares has a bright yellow border. A button showing a gear symbol near the top left of the app shows an access key of 'S'.
Figure 4: An in-progress game of Leaf Sweeper shown using the app's light theme colours. There are fifteen light grey squares, and eleven of those show a black number from 1 to 3. Ten squares have a light green background with dark green symbols of a frog with a question mark, or a leaf. One of those squares has a menu shown above it, containing the text: "Change Query Frog state".
?
It’s next worth considering the action I took relating to the gameplaying experience when a screen reader is running. Let’s consider some topics introduced in the Introduction to UIA - At a Glance series.
?
When you build an app with XAML, the hierarchy of the elements provided to screen readers is largely taken care of by your XAML UI files containing an ordered and logical structure for the UI. Never start forcing elements to appear at specific positions around a page with very large Margin values, (and yes, I have seen this done). Instead create adaptive UI layout leveraging all the helpful support for this that MAUI provides.
The only additional hierarchy-related step I take with the Leaf Sweeper game is that if I’m really, really sure that an element is being exposed to a screen reader and the element is of no interest whatsoever to the players, then I prevent it from being exposed to screen readers. This might make the gameplaying experience more efficient, but this sort of thing must be carefully considered first. An example of where I’ve done this is for the Labels on the items in the CollectionView. Given that the item names themselves provide all the information that players need when they’ve navigated to an item, there’s nothing helpful about explicitly exposing the Labels contained within the items. As such, I set the following property on the Labels.
AutomationProperties.IsInAccessibleTree="False"
Most of my elements’ properties that are exposed for screen readers are taken care of by default, because I used the semantic controls that MAUI provides. For example, my Buttons and CollectionView items have the appropriate IsKeyboardFocusable and HasKeyboardFocus properties set as I move keyboard focus through the game.
However, there are a few properties that I do sometimes customise, because MAUI just can’t know what the most helpful property values would be. And the most important of those properties for my game is the UIA Name property. I’ll discuss that later in the section relating to using Visual Studio’s Accessibility Checker.
?
Because I use the semantic controls that MAUI provides, my UI elements support the patterns expected for each control type, and so there’s no work for me to do here at all. Jolly good.
?
Because I use the semantic controls that MAUI provides, my UI elements raise the events expected for each control type. For example, events are raised when the control gains keyboard focus or when it becomes selected.
However, there is one additional event I do want to explicitly raise myself, and that’s the Notification event. This is an event that must be used with great care, as to use it unwisely could create such distractions to your customers that your app becomes unusable. But in my case, my player can take important action which changes the state of the game, but because keyboard focus doesn’t move throughout that action, by default a screen reader won’t announce anything. As such, I raise a Notification event when a stone is swept in the game, to let players know the outcome of that sweeping action.
领英推荐
Figure 5: The Accessibility Insights for Windows tool reporting the UI Automation FocusChanged and Notification events raised by the Leaf Sweeper game as a game is played. A Notification event is highlighted in the Accessibility Insights for Windows tool and has the following associated text: "Swept stone 3, 2 nearby frogs for stone 3".
Figure 6: A won game of Leaf Sweeper with thirty light grey squares which are either blank or show a number from 1 to 4, and six light green squares showing symbols of a frog and smiley face. The NVDA Speech Viewer feature is next to the app, and shows NVDA's announcements made during gameplay, including: Sweeper Squares list, Query Frog on stone 33 33 of 36, Leaf on stone 31?31 of 36, Swept stone 31, 1 nearby frog for stone 31, Congratulations! You helped all the frogs! Would you like to play another game?
Using Visual Studio’s Accessibility Checker at the Leaf Sweeper app
By the time I learnt of VS’s Accessibility Checker, I’d already taken action to make my app as programmatically accessible as I could, and when I first ran the Accessibility Checker the only failure it reported related to the window element at the root of my UI Automation hierarchy having no name. That can be avoided by setting a Title on the app's Shell in AppShell.xaml, as shown here:
? ? xmlns:res="clr-namespace:GridGames.ResX
? ? Title="{x:Static res:AppResources.GridGames}"
Tip: At the time of writing this, any new .NET MAUI app created in Visual Studio will get a failure reported by the VS Accessibility Checker due to the nameless app window element. So always set a Title on the app Shell and verify that the VS Accessibility Checker then reports no failures against the app. Then as you add UI to the app, frequently run the Accessibility Checker to detect any issues as soon as they exist.
I also found the VS Accessibility Checker reports each issue that it finds twice. But all being well, that issue will be resolved soon.
My fun really started when I began to remove the code I’d added to set custom Name properties on various elements in my app.
The first thing I tried was to remove the following from my Settings button.
SemanticProperties.Description="{x:Static resources:AppResources.PairsSettings}"
?
By doing this, the button’s UIA Name property would revert to being based on the Unicode character I’d set on the Text property of the Button. Visual Studio’s Accessibility Checker can’t assume that a screen reader will announce something helpful for customers when encountering that Unicode character, and so draws attention to the issue with the following report.?
Automation ID: SweeperSettingsButton
Rule: NameExcludesPrivateUnicodeCharacters
Description: The Name property must not contain any characters in the private Unicode range.
How To Fix: Modify the Name property, removing all characters in the range U+E000 to U+F8FF and replace them with meaningful, standard text content.
This is an excellent result by the Accessibility Checker!
Figure 7: An in-progress game of Leaf Sweeper running in the debugger in Visual Studio. to the left in VS is the Live Visual Tree pane containing the button to launch the Accessibility Checker. The bottom pane in Visual Studio is the Accessibility Checker pane, reporting an issue with the Name property of a Button in the Leaf Sweeper game.
Note that I rarely use AutomationIds in my apps but I do expect to be using them a lot more going forward. Your production-quality apps might well make heavy use of AutomationIds already. If Visual Studio’s Accessibility Checker does report an issue with an element with no AutomationId in your app, you could then point Accessibility Insights for Windows at the app, and the element highlight feature in Accessibility Insights might help you pinpoint exactly which element has the issue.
My next experiment was to remove the following from my CollectionView’s item definition.
SemanticProperties.Description="{Binding AccessibleName, Mode=OneWay}"
?
By doing this, the items’ Names would revert to the default Name property provided by MAUI, which is a concatenation of namespace and class names, and of no help at all to customers.
Automation ID:
Rule: NameIsInformative
Description: The Name property of an element should not contain class names like 'Microsoft.*.*' or 'Windows.*.*' as these are not usually informative.
How To Fix: Provide a UI Automation Name property for the element that:
Concisely identifies the element, AND
Does not include the element's class name (such as Microsoft.*.* or Windows.*.*).
?
This is an extremely important result, as it highlights a problem that many apps have historically shipped with. I’d expect that typically if an app shows a collection of items, the items are important. So if a screen reader announces apparent gibberish when moving through the items, that’s a miserable and possibly show-stopping experience for customers. For Visual Studio’s Accessibility Checker to now be able to draw devs’ attention to this problem is a really big deal.
?
And here’s where things got even more interesting for me. It turns out that while experimenting with the Accessibility Checker, it reported an issue that I wasn’t aware of. I really had missed giving an edit control in the game’s Welcome UI an accessible name. ?
Automation ID:
Rule: NameNotNull
Description: The Name property of a focusable element must not be null.
How To Fix: Provide a UI Automation Name property that concisely identifies the element.
?
And the reason why I’d not given the edit control an accessible name? Well, I forgot to. And this made it clear to me that going forward, I’ll be frequently invoking the “Scan for Accessibility Issues” button in VS, fairly frequently running Accessibility Insights for Windows for a deeper examination, and periodically playing end-to-end games with the various input and output methods that Windows makes available to me.
Figure 8: An in-progress game of Leaf Sweeper with a mix of blank squares, squares showing a nearby frog count, squares showing only a leaf, and a square showing symbols of a frog and question mark. Over one square is a menu containing the text "Change Query Frog state". Above the app is the Windows Speech Recognition feature, showing the text: "Right-clicking Leaf on stone 26".
Figure 9: The Leaf Sweeper game running on a Surface laptop, with the screen fully in view and the keyboard partially in view. The screen shows a magnified portion of the Leaf Sweeper game, with only four of the squares in the game fitting on the screen. One magnified square shows a leaf, and has a context menu open above it, saying: “Change Query Frog state”.
Summary
Based on my experiences, I do feel there are a couple of issues with Visual Studio’s new integrated Accessibility Checker that users need to be aware of, (at least that's the case at the time I write this). Specifically, by default two failures get reported with any new MAUI app created in VS due to a nameless window element at the root of the UIA Automation hierarchy. I do feel it's important that devs resolve that issue before beginning to add their own UI to the app, in order for any new failures that appear due to that new UI to really jump out.
That said, I do feel strongly that the introduction of VS’s Accessibility Checker is an important, and necessary, step towards the ultimate goal of devs actually building accessible apps. Some people might say that having to run an accessibility checker shouldn’t be necessary, regardless of whether it’s integrated into an IDE. Some people might say that apps built in Visual Studio should be accessible by default. While I appreciate the sentiment, technology just isn’t at that point yet. To build truly accessible and usable apps today requires human consideration. Without that human consideration, customers are presented with unusable products which prevent them from communicating, gaining and retaining employment, being productive, and yes, enjoying games.
So is it really that much effort to use Visual Studio’s new integrated Accessibility Checker?
I certainly intend to.
Guy