Opportunities for AI in Accessibility

I was completely moved by Joe Dolson’s subsequent article on the crossroads of AI and availability because I found it to be both skeptical about how widespread use of AI is. In fact, I’m very skeptical of AI myself, despite my role at Microsoft as an accessibility technology strategist who helps manage the AI for Accessibility award program. AI can be used in quite creative, inclusive, and accessible ways, as well as harmful, exclusive, and harmful ways, just like with any tool. Additionally, there are a lot of uses in the subpar center.

I’d like you to consider this a “yes … and” piece to complement Joe’s post. Instead of refuting everything he’s saying, I’m pointing out some areas where AI may make real, positive impacts on people with disabilities. To be clear, I want to take some time to speak about what’s possible in hope that we’ll get there one evening. There are, and we’ve needed to address them, like, yesterday.

Other text

Joe’s article spends a lot of time examining how computer vision models can create other words. He raises a number of true points about the state of affairs right now. And while computer-vision concepts continue to improve in the quality and complexity of information in their information, their benefits aren’t wonderful. He argues to be accurate that the state of image research is currently very poor, especially for some graphic types, in large part due to the absence of contextual contexts in which to look at images ( as a result of having separate “foundation” models for words analysis and image analysis ). Today’s models aren’t trained to distinguish between images that are contextually relevant ( which should probably have descriptions ) and those that are purely decorative ( which might not even need a description ) either. However, I still think there’s possible in this area.

As Joe points out, far text editing via human-in-the-loop should be a given. And if AI can intervene and provide a starting point for alt text, even if the rapid reads,” What is this BS?” That’s not correct at all … Let me try to offer a starting point— I think that’s a gain.

If we can specifically teach a design to consider image usage in context, it might be able to help us more swiftly distinguish between images that are likely to be attractive and those that are more descriptive. That will help clarify which situations require image descriptions, and it will increase authors ‘ effectiveness in making their sites more visible.

While complex images—like graphs and charts—are challenging to describe in any sort of succinct way ( even for humans ), the image example shared in the GPT4 announcement points to an interesting opportunity as well. Let’s say you came across a map that was simply the description of the chart’s name and the type of representation it was: Pie graph comparing smartphone usage to have phone usage in US households earning under$ 30, 000 annually. ( That would be a pretty bad alt text for a chart because it would frequently leave many unanswered questions about the data, but let’s just assume that that was the description in place. ) If your website knew that that picture was a pie graph ( because an ship model concluded this ), imagine a world where people could ask questions like these about the creative:

Would more people use smartphones or other types of phones?
How many more are there?
Is there a group of people that don’t fall into either of these containers?
That number, how some?

For a time, the chance to learn more about images and data in this way may be innovative for people with low vision and blindness as well as for those with various forms of color blindness, mental disabilities, and other issues. It could also be helpful in education settings to help people who can see these figures, as is, to understand the data in the figures.

What if you could request your website to make a complicated map simpler? What if you demanded that the line graph be isolated into just one collection? What if you could request your website to transform the colors of the various ranges to work better for type of colour blindness you have? What if you asked it to switch shades in favor of habits? That seems like a chance given the chat-based interface and our current ability to manipulate photos in modern AI tools.

Now imagine a purpose-built unit that was extract the information from that table and turn it to another style. Perhaps it could convert that pie chart (or, better yet, a series of pie charts ) into more usable ( and useful ) formats, like spreadsheets, for instance. That would be incredible!

Matching techniques

When Safiya Umoja Noble chose to write her reserve Algorithms of Oppression, she hit the nail on the head. Although her book focused on the techniques that search engines can foster racism, I believe it to be extremely accurate to say that all laptop models have the potential to intensify issue, discrimination, and hatred. Whether it’s Online usually showing you the latest tweet from a tired billionaire, YouTube sending us into a Q-hole, or Instagram warping our ideas of what organic bodies look like, we know that terribly authored and maintained algorithms are very harmful. A large portion of this is a result of a lack of diversity in the people who design and construct them. However, when these platforms are built with inclusive features in mind, there is real potential for algorithm development to help people with disabilities.

Take Mentra, for example. They serve as a network of employment for people who are neurodivers. Based on more than 75 data points, they match job seekers with potential employers using an algorithm. On the job-seeker side of things, it considers each candidate’s strengths, their necessary and preferred workplace accommodations, environmental sensitivities, and so on. On the employer side, it takes into account each work environment, communication strategies for each job, and other factors. Mentra made the decision to change the script when it came to traditional employment websites because it was run by neurodivergent people. They use their algorithm to propose available candidates to companies, who can then connect with job seekers that they are interested in, reducing the emotional and physical labor on the job-seeker side of things.

More people with disabilities can be used to create algorithms, which can lessen the likelihood that they will harm their communities. That’s why diverse teams are so crucial.

Imagine that a social media company’s recommendation engine was tuned to analyze who you’re following and if it was tuned to prioritize follow recommendations for people who talked about similar things but who were different in some key ways from your existing sphere of influence. For instance, if you follow a group of white men who are not white or aren’t white and who also discuss AI, it might be wise to follow those who are also disabled or who are not white. If you followed its advice, you might be able to understand what is happening in the AI field more fully and nuancedly. These same systems should also use their understanding of biases about particular communities—including, for instance, the disability community—to make sure that they aren’t recommending any of their users follow accounts that perpetuate biases against (or, worse, spewing hate toward ) those groups.

Other ways that AI can assist people with disabilities

I’m sure I could go on and on about using AI to assist people with disabilities, but I’m going to make this last section into a bit of a lightning round. In no particular order:

voice recognition is. Researchers like those in the Speech Accessibility Project are paying people with disabilities for their help in collecting recordings of people with atypical speech. As I type, they are actively recruiting people with Parkinson’s and related conditions, and they intend to expand this to other conditions as the project develops. More people with disabilities will be able to use voice assistants, dictation software, and voice-response services, as well as to use only their voices to control computers and other devices, according to this research.
Text transformation. The most recent generation of LLMs is capable of altering already-existing text without giving off hallucinations. This is incredibly empowering for those who have cognitive disabilities and who may benefit from text summaries or simplified versions, or even text that has been prepared for bionic reading.

The importance of diverse teams and data

Our differences must be acknowledged as important. The intersections of the identities we exist in have an impact on our lived experiences. These lived experiences—with all their complexities ( and joys and pain ) —are valuable inputs to the software, services, and societies that we shape. Our differences must be reflected in the data we use to develop new models, and those who provide it need to be compensated for doing so. Stronger models can be created using inclusive data sets, which lead to more equitable outcomes.

Want a model that doesn’t demean or patronize or objectify people with disabilities? Make sure that you include information about disabilities that is written by people who have a range of disabilities and that is well represented in the training data.

Want a model that uses ableist language without using it? You may be able to use existing data sets to build a filter that can intercept and remediate ableist language before it reaches readers. Despite this, AI models won’t soon replace human copy editors when it comes to sensitivity reading.

Want a copilot for coding that provides recomprehensible recommendations after the jump? Train it on code that you know to be accessible.

I have no doubts about how dangerous AI can and will be for people today, tomorrow, and for the rest of the world. However, I also think that we can acknowledge this and make thoughtful, thoughtful, and intentional changes in our approaches to AI that will reduce harm over time as well. Today, tomorrow, and well into the future.

Many thanks to Kartik Sawhney for supporting the development of this article, Ashley Bischoff for providing me with invaluable editorial support, and, of course, Joe Dolson for the prompt.