Considerations Around Building an Accessible Multi-platform App

This article describes considerations around building an accessible app, using an app built with .NET MAUI as an example. The app is available at all of the Apple, Google, and Microsoft Stores, and its source code is publicly available.

Feedback on the accessibility of the app would be very much appreciated, to help me learn how I should prioritise the future enhancements I make to it.

?

Introduction

My journey of exploration around accessible app technology never ends, and here I continue along its winding path with, for the first time, a single app that runs on all of iOS (iPhone, iPod), Android, and Windows. This has been made possible thanks to the very exciting release of .NET MAUI 7.0. In this article I’ll describe some of my considerations while building the app, in the hope that this might help you as you consider the accessibility of your own apps. Many of these considerations I’ve already mentioned in tweets as I was building the app, via @gbarkerz.

The app today certainly has some significant bugs, some of which I should be able to fix myself, and some I’ll need to wait for fixes from the underlying technology that the app uses. That said, please don’t let those bugs deter you from trying out the app and letting me know how you feel I can improve it, so that I can make the games in the app more usable and more fun to play. The app is a work in progress, and while it’s far from perfect, I’ve reached a point now where your feedback would be very helpful indeed.

Later in this article I’ll describe some of the specific implementation work I did in the app to enhance its accessibility. Before that however, I’ll include some thoughts on design and use of semantic controls in the app. Without careful consideration of design and semantics first, there’s little point in discussing what a dev might do to tune an app’s accessibility.

The source code for the app is available at Grid Games source. Please note that that code is only intended to demonstrate specific implementation details relating to accessibility. It does not demonstrate any other best practices around building a .NET MAUI app.

The app itself is available from the following Stores:?

• Apple Store. This is only available for iPhone and iPod, because I don’t have an iPad and I couldn’t get the iPad simulator to work in Visuals Studio for Mac.
• Google Play Store. This does not yet support use with a keyboard.
• Microsoft Store.

For an introduction into how to play the Squares, Pairs, and Where’s WCAG? games contained in the app, please refer to the in-app Help content for each game.

?

Personalising the games

I built the app with the intent of learning how I might build more accessible puzzle game experiences, and then share those learnings. My hope is that the resulting games can also be a relaxing way for people to pass the time. I know they have been for me.

Importantly, the pictures that the games show can be customised to make the games more personal, and so more enjoyable. For example, perhaps you might customise the Pairs game to show pictures of friends and family.

In this article, the custom pictures shown are the flags of some of the football teams playing in the 2022 men’s World?Cup. In selecting these flags, I took care not to present two flags where the only differences between them are the colours used.

?

?

?

Design

If an app has any hope of being accessible, useable, and a delight for everyone to use, then it must be designed with inclusion in mind. Trying to retroactively fix all the accessibility bugs in a poorly designed app might resolve a few bugs reported by an automated test tool, but the app’s still going to be broken for real people.

Thankfully many people today understand the need to consider all customers when designing apps, and there are a growing number of helpful resources available. For example, Microsoft’s Inclusive Design site. And for a quick introduction into some interesting design topics, visit Accessible Design is Good Design – At a Glance, and its accompanying Part 2 and Part 3.

Those “At a Glance” resources above mention a number of interesting points to consider, and some of those points are not respected by my app. But perhaps some of them are. Let’s consider the first three of Schneiderman’s Eight Golden Rules of User Interface Design.

1. Strive for consistency. The app has three puzzle games which each present a grid of four-by-four squares. The methods of navigating around the grids and invoking squares within the grids are consistent across the different games.
2. Enable frequent users to use shortcuts. All controls within the Windows version of the app have associated keyboard shortcuts, and a press and release of the Alt key will reveal those shortcuts. And for controls that exist in all of the games (for example, the games’ Settings buttons) the keyboard shortcut for the controls is consistent across games.
3. Offer informative feedback. This is an area where I’ve not been consistent, and so is an opportunity where things can be improved. For example, when playing the Squares game, if a square is invoked and it cannot move, a screen reader will announce: “A move is not possible from here.” However, there’s no visual equivalent of this announcement. Should there be?

?

Another interesting point to consider is how well an app design will respect all the accessibility standards included in the European Standards EN 301 549. If a design respects those standards, and a dev builds the app to that design, then the majority of the traditional accessibility bugs will be avoided. If, however, the app design includes such things as text colours which have insufficient contrast against the text background, or timeouts which cannot be controlled by the customer, or no keyboard navigation paths defined at all, then this is an incomplete design which will inevitably lead to misery for the dev when the bugs come rolling in. It’ll also lead to misery for customers given that we all know many of those accessibility bugs won’t get fixed.

My app is partially compliant with those international standards, but I have some more work to do yet. Some specific points are as follows:

1. Colour contrast. As far as I know, all text shown in the app has sufficient contrast against its background, as does non-text visuals such as keyboard focus feedback. Importantly, this is true when the app is showing its light theme and its dark theme.
2. Timeouts. As I said in this tweet, I find the topic of supporting customers in their desire to control timeouts particularly interesting. My app’s compliant around timeouts by not having any timeouts at all in the app.
3. Large text. This is an area where I still have work ahead. I’ve deliberately not sized any text elements, or containers of text, using fixed widths and heights. By doing that, I’ve reduced the risk of text being clipped as my customers change the size of text shown on the device via the device’s system settings. I’ve used the ScrollView control such that if content is too large for a single screen, the full content can be scrolled into view. That said, I do know of cases where content is clipped at very large text, and I need to consider my options there. As I do that, I’ll be keeping such things as the FlexLayout and LineBreakMode in mind.

?

So, the message here is: Please design your app such that it adheres to the Inclusive Design methodology and is compliant with the European Standards EN 301 549. With that design in place, you can feel confident that you’re treating your customers with respect.

Semantic controls

It’s next the dev’s job to turn that great design into something that all customers can benefit from. With a poor design, a dev just can’t do that. With a great design, a dev can still build an unusable app if they choose to.

There’s only one point here that I want to make. That is: Devs please use the UI framework’s controls which are the best semantic match for the UI. This is so very important, and without following this guideline, there’s a good chance the app will be unusable to many customers.

For example, say your app presents an image and a button. The best semantic match for the image would be the Image control, and for the button it would be the Button control, and so for the sake of your customers, those are the controls that must be used.

Now, you may feel: What difference does it make? A Tapped event handler can be attached to an Image control, and so it behaves just like a button. But the trouble is, it doesn’t. It’s a natural tendency for devs to build UI that they themselves can use. So if the dev can consume the visuals shown by an image on the screen, and can prod the image with their finger, then it’s a fully functional button. But what about all the customers who interact with the UI through other means? What about your customers who use a keyboard, or speech, or eye control, and do not use touch to input at the device? And what about all the customers who consume information relating to the controls via a screen reader’s audio or braille device rather than the visuals shown on the screen?

Your first reaction might be that to support so many types of input or output, surely it must be tons of work, and how on earth will you find time for all that work? The great news is that if you use the type of control that’s the best semantic match for the UI, then you’re already well on the way to supporting all types of input and output. By adding a Button control for some UI that’s semantically a button, you’ll often get all of the following greatest hits automatically:

• Customers can move keyboard focus to it.
• Customers know when they’ve moved keyboard focus to it, regardless of the current theme.
• Customers can fully interact with it via the keyboard, including to toggle or expand the button if necessary.
• Text shown on the button is shown with significant contrast against its background.
• Customers’ expectations are met when magnification is turned on.
• A screen reader can navigate to it via the screen reader’s own navigation mechanism.
• A screen reader can access the text on the button, and also access its current state if it can be toggled or expanded.
• And many more.

?

If a control is used which isn’t the best semantic match for the UI, at least some of the above won’t be true, and many customers won’t be able to use the control.

So, the message here is: Please build your app with controls that are the best semantic match for the UI. With that initial implementation in place, you can feel confident that you’re treating your customers with respect.

?

?

?

Additional essential considerations

Ok, so your app has a great inclusive design and you’re going to build it using controls that are the best semantic match for the UI, and you’re well on the way to delivering the experiences that your customers need. Next we need to consider what additional work might be required such that the final experiences you deliver are both usable and delightful.

Supporting all types of input

If at all possible, we want to support all our customers regardless of their method of inputting at the device. For example, touch, mouse, keyboard, and each of those with or without a screen reader running. And of course eye control,?speech, and switch device. But who knows exactly what input events will be received by your app for each of those types of input? I certainly don’t. So I tend to find it’s a case of experimenting with different types of input, and building an app which accounts for whatever events arrive.

For example, does the app receive a Tapped gesture event or a SelectionChanged event when using some input device at the items in a CollectionView? And what happens when one type of input generates one of those events, and another type of input generates the other event, and yet a third type of input generates both events?

My app handles both the Tapped gesture event and the SelectionChanged event, and it has some rather dodgy-looking code to account for both events arriving around the same time when I only want to react once to some input. Basically when both types of events arrive around the same time, I ignore the second of the two. Yeah, it’s pretty ugly I know, but it enables the app experience I’m after. I’ll revisit this at some point and try to make it more robust.

It's also worth noting that I added some Windows-only code relating to handling keyboard input. This means the app can react exactly how I’d like it to in response to a press of Space, Enter, or a Function key. This platform-specific code seemed unavoidable at the time I built the app, but I assume at some point I can replace it with some cross-platform code which will support keyboard input across all of iOS, Android, and Windows.

?

?

Light and Dark Theme Support

.NET MAUI makes it straightforward for the app to show specific colours based on whether it’s showing its Light Theme or Dark Theme. Just search for “AppThemeBinding” across the various XAML files in the app source to see how I had the app support the different themes. Lots of helpful related details are at Respond to system theme changes.

As I mentioned in this tweet, I’ve not had any luck customising the experience when a Windows High Contrast theme is active, so maybe that’s something I’ll revisit at some point.

?

?

?

Controlling the accessible names of UI elements

.NET MAUI takes very helpful action such that if text is set on a control like a Button, that text is accessible to a screen reader. This means that in some cases, controls are accessible by default. However, in practice a dev will always need to take some action to customise the app experience in order to make an app usable to all customers. For example, to set an accessible name on a button which only shows an image and no text. This customisation is achievable by leveraging the .NET MAUI SemanticProperties class. Related details are at Build accessible apps with semantic properties.

Searching the app’s source code for “SemanticProperties” shows all the places where I’ve leveraged this very important class. Below I’ve described some of the specific considerations I had while building the app.

Note: For an introduction into the technology used to power programmatic accessibility on Windows, please visit my UI Automation: An Incomplete Guide for UI builders, and to examine the programmatic accessibility of your own Windows app, use the Accessibility Insights for Windows tool.

My original hope was that the accessible hierarchy of the upturned cards in the Pairs game would be one where each card was represented as a list item whose name identifies the card, and that would be the parent of an image whose name describes the picture shown on the card. That worked great on Windows with the Narrator and NVDA screen readers, as it meant that once my customer has reached a card of interest, they can use the screen reader’s own navigation method to move to the contained image to learn more. However, I couldn’t find a way to navigate like that using touch with the VoiceOver and Talkback screen readers, so for iOS and Android, the card itself exposes both the name and description data, with the description accessible to the screen reader as what the screen reader considers to be “Hints”.

Note: I have outstanding work around the management of two different UI Automation hierarchies. Figure 15 below shows the Pairs game running on Windows, and shows that most of the squares have the desired UI Automation hierarchy of a list item containing an image, and the image’s name describes the picture. However, one list item has a name which contains both the identifying name of the square and its description. So I’ll prioritise tidying this up so that a list item contains both the name and description only on iOS and Android.

By default I try to give semantic containers accessible names if customers would find information about UI grouping helpful. However, on iOS the VoiceOver screen reader wouldn’t navigate into a named container, and that left the content of the container inaccessible. So I now have some iOS-specific code in the app to remove the naming of some containers.

I tried wherever possible to bind the accessible data to object properties. Overall this seemed to work well, but at the time of building the app I didn’t find the UI was always updated as expected following a change to the underlying property. So I ended up making multiple changes to the property each time a change was required, and that seemed to nudge the UI enough such that it got updated to reflect the change. I’m sure I’ll be cleaning that up at some point.

?

Custom screen reader announcements

By default, I do not want my app to trigger any custom screen reader announcements during gameplay. Rather I’d prefer that the screen reader makes whatever announcements it feels are most helpful to my customers, based on the various data made accessible by my UI. For me to decide that additional announcements should be made, is a very risky thing. To do so might mean that customers are blocked from accessing the information they really need, because my custom announcements are interfering with the output of that critical information. Who am I to decide what all my customers would prefer without some serious consultation?

In order to get the ball rolling on some discussion around the question of what custom announcements might be helpful, I have added some to the app. I was most interested in situations where there would be no announcement at all without a custom announcement, and yet where I felt something important had occurred.

For example, when playing the Pairs game, to turn two unmatched upturned cards back down, another card is tapped. In that case, there’s no change of focus and the state of the focused element does not change, and as such, no screen reader announcement is made by default. So here I added a custom announcement of, “Unmatched cards turned back”. Similarly in the Squares game, if a square is tapped in an attempt to move it, and it does not lie next to the empty square, the tapped square will not move and there’s no change of focus. In that case I added a custom announcement of, “A move is not possible from here”.

Perhaps it would be preferable if the app didn’t include these custom announcements, or the announcements could be made more helpful, and this is where I’m hoping to get your feedback so that I can learn.

The custom announcement feature in the app is an area where I know I still have work to do, due to the classic challenge of not having the custom announcements interrupted by other announcements relating to focus changing. I found on iOS, VoiceOver would often repeat the name of the already-focused element in such a way that my custom announcements never got announced. In order to try to reduce this problem, I introduced a slight delay in starting my custom announcements, in the hope of preventing them from being interrupted. That did indeed seem to fix the issue on iOS, but at the same time degraded the custom announcements on Android.

I think it might take me a while to arrive at the best experience I can around custom announcements on all of iOS, Android, and Windows, so please forgive an inconsistent announcement experience across platforms today, and I’ll try to improve things as best I can.

There are some other interesting points I could be including here around my experiences building the app, but for a full history of what was going through my mind, please review my tweets over the last few months. I tended to tweet about most of the surprises and challenges I hit.

?

Summary

I never thought I’d build an accessible app which can run on all of iOS, Android, and Windows. The current version of the app has some bugs, and some flaws in its design, but for it to be as usable as it is in what I consider to be its first release blows me away. I’m so impressed with the .NET MAUI team for all the work they’ve put in to support app builders in their efforts to build great experiences for all their customers.

I know I have a lot of work ahead around improving and fixing the experiences in my app, but I find this such an encouraging start. I am looking forward to continuing this exploration next year, but I really do need your help. Please give the app a whirl on your device of choice, and let me know how I need to prioritise my work.?Without your input, it’s likely that this app’s journey will end soon, but if it would be practical for you to provide feedback, I can continue learning, and sharing those learnings with everyone.

If you do have feedback around bugs and suggestions for the app, feel free to log something up at Grid Games Issues.

Here’s to continued progress in 2023!

All the best everyone.

Guy