Building Inclusive Systems for Individuals with Disabilities

Across 微软 , employees with disabilities are using Copilot to support productivity and simplify tasks. Generative AI, in particular, provides an opportunity to all organizations and enterprises to help bridge the disability divide.??

“When I think about generative AI, I think about things that I’ve done in my 20-plus year career in accessibility that we’ve not been able to do before now,” says Microsoft Senior Support Escalation Engineer Jeremy Curry . “And some of these things are literally lifechanging.”?

Inclusivity is at the heart of our responsible AI progress. Yet too often, disability is?underrepresented ?or?incorrectly categorized ?in the large datasets required to train AI models. A recent paper ?from Microsoft Research found that disability objects, such as a braille device, were recognized ~30 percent less accurately by AI models. ?

This disability data desert can?limit the utility of a technology, ?strengthen existing stereotypes and magnify bias. We’re committed to partnering with the disability community and other organizations to build inclusive datasets and design solutions specifically for disability communities. Here are a few of the ways we’re doing that:?

Training AI Systems on Disability Representative Data?

Our recently announced collaboration with Be My Eyes ?is designed to improve the quality of datasets used to train AI systems by incorporating high-quality, disability representative data of the live experience of the blind and low vision community, thereby helping make Microsoft AI models more inclusive.??

Transparency and user control are the guiding principles for data privacy in this agreement. Be My Eyes will remove personal information before sharing the data, and we will then use the data to improve the accuracy and precision of scene understanding and descriptions, with the goal of increasing the utility of AI applications for this community.?

Our collaboration with Be My Eyes began in 2017 when their app was integrated into the Disability Answer Desk support experience to give customers more efficient technical support. Last year, Microsoft was the first to?pilot their new ‘Be My AI’ , with AI helping to solve technical support issues. Collaborations like these allow us to improve representation and diversity of AI data and drive more authentic inclusion of disability in our technology.?

Designing for Vision Loss through AI for Good?

In the 2014 Microsoft Hackathon, Saqib Shaikh , then a software engineer, began experimenting with how AI could help the 340 million people in the world like himself who are blind or have low vision. Three years later, in 2017, Microsoft launched the Seeing AI app to provide blind and low-vision individuals with more information about the world around them using their smartphones. In 2018, Seeing AI was awarded the Helen Keller Achievement Award from the American Foundation for the Blind. Saqib, now a Principal Engineering Manager, continues to lead the Seeing AI team, demonstrating how disabled talent is driving innovation with AI.?

Since its launch, Seeing AI has helped the blind community complete more than 10 million tasks. Seeing AI can recognize friends, provide information about products at grocery stores with its barcode scanner, and read descriptions of images and text found on social media.??

Our AI for Good Lab created a machine learning model using more than 24,000 banknotes spanning 17 currencies and 112 denominations in real-world scenarios, in which part of the banknote image might be blurred or poorly lit. The open dataset developed by researchers, BankNote-Net , drastically improved the currency recognition feature within Microsoft’s Seeing AI app. Seeing AI now supports recognition of 17 different currencies and averages 100,000 monthly users in 19 countries, helping individuals who are blind or low vision navigate the world around them with greater ease.??

Improving the Accessibility of Voice Recognition Technology?

In 2022, Microsoft and other leading tech companies announced our participation in the Speech Accessibility Project, an initiative led by the 美国伊利诺伊大学香槟分校 designed to improve the diversity of speech samples used to train speech recognition tools such as voice assistants and translation tools. Without diverse, representative data, machine learning models cannot learn diversity of speech, limiting customer scenarios where this technology can bring benefits. As of the end of June 2024, the Speech Accessibility Project had shared 235,000 speech samples with us and other tech companies. This data has already led to marked improvement in an automatic speech recognition tool trained by the University of Illinois as part of the research —before incorporating this data, the tool misunderstood speech 20% of the time. With this data, that number has dropped to 12%.??

Ensuring AI tools are inclusive and representative of all who use it is one of the guiding principles for how we develop AI responsibly. We believe accessible technology is a fundamental right that can unlock opportunities in every part of society. When designed inclusively and responsibly with the disability community, generative AI is one of the most powerful tools we have to deliver on that potential.??