Opportunities for AI in Accessibility

I was completely moved by Joe Dolson’s subsequent article on the crossroads of AI and availability because I found it to be both skeptical about how widespread use of AI is. In fact, I’m very skeptical of AI myself, despite my role at Microsoft as an accessibility technology strategist who helps manage the AI for Accessibility award program. As with any device, AI can be used in very positive, equitable, and visible ways, as well as in destructive, unique, and harmful ways. Additionally, there are a lot of functions in the subpar center.

I’d like you to consider this a “yes … and” piece to complement Joe’s post. I’m just trying to reject what he’s saying, but I’m just trying to give some context to initiatives and opportunities where AI can make a difference for people with disabilities. To be clear, I’m not saying that there aren’t true threats or pressing problems with AI that need to be addressed; there are, and we’ve needed to address them, like, yesterday; instead, I want to take a moment to talk about what’s possible so that we can get there one day.

Other text

Joe’s article spends a lot of time addressing computer-vision types ‘ ability to create alternative words. He raises a lot of valid points about the state of the world right now. And while computer-vision concepts continue to improve in the quality and complexity of information in their information, their benefits aren’t wonderful. He argues to be accurate that the state of image research is currently very poor, especially for some graphic types, in large part due to the absence of contextual contexts in which to look at images ( as a result of having separate “foundation” models for words analysis and image analysis ). Today’s models aren’t trained to distinguish between images that are contextually relevant ( should probably have descriptions ) and those that are purely decorative ( couldn’t possibly need a description ) either. However, I still think there’s possible in this area.

As Joe points out, far text editing via human-in-the-loop should be a given. And if AI can intervene to provide a starting place for alt text, even if the rapid might say What is this BS? That’s not correct at all … Let me try to offer a starting point— I think that’s a gain.

If we can specifically station a design to examine image usage in context, it might help us more quickly determine which images are likely to be elegant and which ones are likely to need a description. That will clarify which situations require image descriptions, and it will increase authors ‘ effectiveness in making their sites more visible.

While complex images—like graphs and charts—are challenging to describe in any sort of succinct way ( even for humans ), the image example shared in the GPT4 announcement points to an interesting opportunity as well. Let’s say you came across a map that merely stated the chart’s name and the type of representation it was:” Pie chart comparing smartphone use to have phone usage in US households making under$ 30, 000 annually.” ( That would be a pretty bad alt text for a chart because it would frequently leave many unanswered questions about the data, but let’s just assume that that was the description in place. ) If your website knew that that picture was a pie graph ( because an onboard model concluded this ), imagine a world where people could ask questions like these about the creative:

Do more people use smartphones or other types of smartphones?
How many more?
Is there a group of people that don’t fall into either of these buckets?
What number is that?

For a moment, the chance to learn more about images and data in this way could be revolutionary for people who are blind and low vision as well as for those with various forms of color blindness, cognitive disabilities, and other issues. Putting aside the realities of large language model ( LLM) hallucinations, where a model just makes up plausible-sounding “facts,” It could also be useful in educational contexts to help people who can see these charts, as is, to understand the data in the charts.

What if you could ask your browser to make a complicated chart simpler? What if you asked it to separate a single line from a line graph? What if you could ask your browser to transpose the colors of the different lines to work better for form of color blindness you have? What if you demanded that it switch colors in favor of patterns? That seems like a possibility given the chat-based interfaces and our current ability to manipulate images in today’s AI tools.

Now imagine a purpose-built model that could extract the information from that chart and convert it to another format. For instance, it might be able to convert that pie chart (or, better yet, a number of pie charts ) into more usable ( and useful ) formats, like spreadsheets. That would be incredible!

Matching algorithms

When Safiya Umoja Noble chose to put her book Algorithms of Oppression, she hit the nail on the head. Although her book focused on the ways that search engines can foster racism, I believe it’s equally true that all computer models have the potential to foster conflict, prejudice, and intolerance. Whether it’s Twitter always showing you the latest tweet from a bored billionaire, YouTube sending us into a Q-hole, or Instagram warping our ideas of what natural bodies look like, we know that poorly authored and maintained algorithms are incredibly harmful. A large portion of this is attributable to the lack of diversity in those who create and shape them. However, when these platforms are built with inclusive features in mind, there is real potential for algorithm development to help people with disabilities.

Take Mentra, for example. They serve as a network of employment for people who are neurodivers. They match job seekers with potential employers using an algorithm based on more than 75 data points. On the job-seeker side of things, it considers each candidate’s strengths, their necessary and preferred workplace accommodations, environmental sensitivities, and so on. On the employer side, it takes into account each work environment, communication issues relating to each job, and other factors. Mentra made the decision to change the script when it came to typical employment websites because it was run by neurodivergent people. They use their algorithm to propose available candidates to companies, who can then connect with job seekers that they are interested in, reducing the emotional and physical labor on the job-seeker side of things.

When more people with disabilities are involved in developing algorithms, this can lower the likelihood that these algorithms will harm their communities. Diverse teams are crucial because of this.

Imagine that a social media company’s recommendation engine was tuned to analyze who you’re following and if it was tuned to prioritize follow recommendations for people who talked about similar things but who were different in some key ways from your existing sphere of influence. For instance, if you followed a group of nondisabled white male academics who spoke about AI, it might be advisable to follow those who are disabled, aren’t white, or aren’t men who also speak about AI. If you followed its recommendations, you might learn more about what’s happening in the AI field. These same systems should also use their understanding of biases about particular communities—including, for instance, the disability community—to make sure that they aren’t recommending any of their users follow accounts that perpetuate biases against (or, worse, spewing hate toward ) those groups.

Other ways that AI can assist people with disabilities

If I weren’t attempting to combine this with other tasks, I’m sure I could go on and on, giving various examples of how AI could be used to assist people with disabilities, but I’m going to make this last section into a bit of a lightning round. In no particular order:

Voice recognition. Researchers like those in the Speech Accessibility Project are paying people with disabilities for their help in collecting recordings of people with atypical speech. As I type, they are actively recruiting people with Parkinson’s and related conditions, and they intend to expand this to other conditions as the project develops. More people with disabilities will be able to use voice assistants, dictation software, and voice-response services as a result of this research, which will result in more inclusive data sets that will enable them to use their computers and other devices more easily and with just their voices.
Text transformation. LLMs of the current generation are quite capable of changing text without creating hallucinations. This is incredibly empowering for those who have cognitive disabilities and who may benefit from text summaries, simplified versions, or even text that has been prepared for Bionic Reading.

The importance of diverse teams and data

We must acknowledge the importance of our differences. The intersections of the identities we live in have an impact on our lived experiences. These lived experiences—with all their complexities ( and joys and pain ) —are valuable inputs to the software, services, and societies that we shape. The data we use to train new models must be based on our differences, and those who provide it to us need to be compensated for doing so. More robust models are produced by inclusive data sets, which promote more justifiable outcomes.

Want a model that doesn’t demean or patronize or objectify people with disabilities? Make sure that the training data includes information about disabilities written by people with a range of disabilities.

Want a non-binary language model? You may be able to use existing data sets to build a filter that can intercept and remediate ableist language before it reaches readers. Despite this, AI models won’t soon replace human copy editors when it comes to sensitivity reading.

Want a copilot for coding that provides recommendations that are accessible after the jump? Train it on code that you know to be accessible.

I have no doubt that AI has the potential to harm people today, tomorrow, and long into the future. However, I also think that we can acknowledge this and make thoughtful, thoughtful, and intentional changes in our approaches to AI that will reduce harm over time as well. Today, tomorrow, and well into the future.

Thanks to Kartik Sawhney for assisting me with writing this article, Ashley Bischoff for her invaluable editorial assistance, and of course Joe Dolson for the prompt.