Left to my own devices, I probably would (play Sudoku)

This article discusses a variety of different input and output methods which can be used to play a Sudoku game.

Introduction?

I was recently asked why I keep exploring the accessibility of a Sudoku game. There are a few reasons, but one real motivator was learning that popular versions of Sudoku from a couple of large, successful companies were inaccessible. It seemed that somehow, something in this world was very broken. So my hope in building the game has been that by demonstrating how practical it is to build accessible Sudoku experiences, more discussion around the topic might be generated. Enough discussion perhaps that might lead one day to those large, successful companies updating their Sudoku games so that the games can be enjoyed by more people, regardless of how those people work at their devices.

In the meantime, I’ve continued exploring how a Sudoku game might be played with a variety of different input and output methods. This article mentions some of the different ways I’ve tried myself when running the Windows version of the game.

The game is freely available at Grid Games for Windows.

Feedback is always welcome on the game, and if you have thoughts on how the game could be made more fun for you or someone you know, please do let me know.

Ways of playing the Sudoku game

When it comes to supporting more player choices, I expect many Sudoku apps go as far as offering a light mode and dark mode, but little beyond that. So below I’ll mention a few other things that we might consider, as we work towards building an app that everyone can enjoy.

Keyboard

It might seem that keyboard usage is so fundamental that it doesn’t need mentioning here, but I’ve encountered so many apps that can’t be used with the keyboard. So we really need all those devs out there who today only test their apps by clicking with a mouse or prodding with their finger, to also test their apps’ end-to-end experiences with only the keyboard as their input device.

Many people for many reasons only use the keyboard. For example, to control a screen reader.

As far as I know, my Sudoku game can be efficiently played with the keyboard.

Screen readers

My Sudoku game has been built using .NET MAUI, and .NET MAUI provides enough support for devs around accessibility for the devs to deliver efficient Sudoku game experiences. So all devs, please consider how you can leverage all that the UI framework has to offer around accessibility.

I received some very helpful feedback relating the screen reader experience in the Sudoku game, including thoughts on how certain keyboard shortcuts could make the game experience much more efficient.

A video of the Sudoku game being played with the NVDA screen reader is at Sudoku being played with the NVDA screen reader.

The Sudoku game can also be played using the Windows Narrator screen reader. A video showing Narrator being used to play another of the games in the Grid Games app, is at The Pairs Game being played with Windows Narrator.

Magnification

Some forms of magnification rely on UI elements raising certain focus-related events for the magnifier to react to. If your app uses standard controls provided by the UI framework, the magnifier will often work by default.

Switch device

A switch device can be used to control the Windows On-Screen Keyboard (OSK), via the OSK’s Scan Mode. Interestingly some of the keyboard shortcuts that I’d added to enhance the screen reader experience, made the switch experience much more efficient too. In fact I added some more keyboard shortcut support (for example, to support Home, End, PageUp, PageDown) to improve the switch experience even further.

A video showing Sudoku being played using a switch device, is at Playing a Sudoku game with a single switch.

Speech

One of the interesting questions for a dev when considering how to support many different input devices and assistive technologies, is what are the different types of input event arriving at the app? Could one input be a click, another a selection, and yet another be both a click and a selection? For my exploratory app, it was fine for me to incrementally add code to generate the experiences I wanted, but a commercial app would want to consider all the different input events up front, and code accordingly.

With respect to speech input, I only tested my app with Windows Speech Recognition (WSR), but it’s possible that other assistive technologies would interact with the app in a different way. For example, I suspect WSR is trying to programmatically invoke items in the grid of squares, but my .NET MAUI CollectionView’s squares don’t support that. Rather they support being selected. To account for this, my exploratory app has an invokable button beneath each square for WSR to invoke, which leads to the square being selected.

A video showing Sudoku being played using Windows Speech Recognition, is at Playing a Sudoku Game with Speech.

Note that Windows Speech Recognition, along with many other assistive technologies running on Windows, will interact with the apps like my Sudoku game using Windows UI Automation. (For an introduction into UI Automation, visit my two-part UI Automation: An Incomplete Guide for UI builders.) So to build an accessible app on Windows, it’s essential to run the Accessibility Insights for Windows tool and learn exactly how your app’s UI is representing itself for assistive technologies.

The following figure shows the Accessibility Insights for Windows tool reporting the programmatic hierarchy of the Sudoku game, and it also shows the tool programmatically controlling the Sudoku game UI.

Eye Control

I’ve experimented in the past with Windows Eye Control, and successfully played one of my earlier games (Sa11ytaire) using only my eyes. I’d hoped to repeat this test with the Sudoku game, but no longer had an eye tracker. The Tobii Eye Tracker 4C is at the top of the list of supported devices at Set up an eye tracking device, but I couldn’t find that device available, so bought a Tobii Eye Tracker 5 instead. I then learnt that that device cannot be used to control Windows and is not compatible with the Microsoft Gaze Interaction API, so sadly I had to return it.

As such, I’ve not been able to play my Sudoku game with eye control. I do hope it works.

A video showing my old Sallytaire game being played using Windows Eye Control, is at Playing Sa11ytaire with Windows 10 eye control.

Xbox Adaptive Controller

When I got out my Xbox Adaptive Controller (XAC), I wondered if I’d be able to use it to control the Windows OSK’s Scan mode. But that’s not possible given that the XAC doesn’t natively generate mouse or keyboard input. I believe it would be possible to control the OSK using the Microsoft Adaptive Hub connected to another device like the Microsoft Adaptive Dual Button, but I’ve not tried that out myself.

So as a proof-of-concept, I added a little native support for the XAC to the Sudoku game. It’s very much a proof-of-concept, as what I did doesn’t support moving focus into the grid of squares, nor dismissing the “Congratulations” window at the end of a game. But it was sufficient to support moving around the grid and inserting numbers into blank squares.

I couldn’t find a way of having my .NET MAUI app react to XAC button events as I really wanted to, so I ended up having the app poll for the state of the XAC buttons. I really, really didn’t want to do that, but this is a proof-of-concept after all.

A video showing Sudoku being played using an Xbox Adaptive Controller, is at Playing a Sudoku Game with an Xbox Adaptive Controller.

Touch and Mouse

Despite many other apps thinking only of touch and mouse when it comes to input, these forms of input were way down my list of considerations. I still wanted it to be possible to play the game with touch and mouse, but really, there are already a million and one Sudoku games that can be played like that.

So today when using touch or mouse to play my Sudoku game, an on-screen keyboard would also be used to select numbers to be inserted in the blank squares. When playing the Android or iOS versions of the game, the game will present its own number pad popup for players to select from.

Summary

This exploration into delivering accessible Sudoku experiences really has been a fascinating one for me, but it’s possible that it’s now run its course. Exploring different puzzle game experiences over the last eighteen months, with multiple UI frameworks and multiple devices, has been a real learning experience for me. That said, I’ve not received feedback for a while, and downloads of the app have dried up, so it doesn’t seem that the way I’ve tackled the inaccessible Sudoku problem has been effective.

If I do get more feedback at some point, I can hopefully react to that, but otherwise, I think it may be time to start looking for new projects.

Thanks everyone who has sent feedback – it kept me going!

All the best,

Guy