Opportunities for AI in Accessibility

I was completely moved by Joe Dolson’s subsequent article on the crossroads of AI and convenience, both in terms of the suspicion he has regarding AI in general and how many people have been using it. Despite my role at Microsoft as an affordability technology tactician who helps manage the AI for Accessibility grant program, I’m very skeptical of AI myself. As with any tool, AI can be used in quite productive, equitable, and visible ways, and it can also be used in dangerous, unique, and dangerous ones. And there are a lot of uses for the poor midsection as well.

I’d like you to consider this a “yes … and” piece to complement Joe’s post. I’m just trying to reject what he’s saying, but I’m just trying to give some context to initiatives and opportunities where AI can make a difference for people with disabilities. To be clear, I’m not saying that there aren’t true threats or pressing problems with AI that need to be addressed—there are, and we’ve needed to address them, like, yesterday—but I want to take a little time to talk about what’s possible in hope that we’ll get there one day.

Other words

Joe’s article spends a lot of time examining how computer vision models can create other words. He raises a lot of valid points about the state of the world right now. And while computer-vision concepts continue to improve in the quality and complexity of information in their information, their benefits aren’t wonderful. He argues to be accurate that the state of image research is currently very poor, especially for some image types, in large part due to the absence of contextual contexts in which to look at images ( as a result of having separate “foundation” models for words analysis and image analysis ). Today’s models aren’t trained to distinguish between images that are contextually relevant ( that should probably have descriptions ) and those that are purely decorative ( which might not need a description ) either. Nonetheless, I still think there’s possible in this area.

As Joe mentions, human-in-the-loop publishing of alt word should definitely be a factor. And if AI can intervene and provide a starting point for alt text, even if the rapid reads,” What is this BS?” That’s certainly correct at all … Let me try to offer a starting point— I think that’s a win.

If we can specifically station a design to examine image usage in context, it might help us more quickly determine which images are likely to be elegant and which ones are likely to need a description. That will help clarify which situations require image descriptions, and it will increase authors ‘ effectiveness in making their sites more visible.

The image example provided in the GPT4 announcement provides an interesting opportunity as well, even though complex images like graphs and charts are challenging to describe in any kind of succinct way ( even for humans ). Let’s say you came across a map that was simply the description of the chart’s title and the type of representation it was: Pie map comparing smartphone usage to have phone usage in US households earning under$ 30, 000 annually. ( That would be a pretty bad alt text for a chart because it would frequently leave many unanswered questions about the data, but let’s just assume that that was the description in place. ) Imagine a world where people could ask questions about the vivid if their browser knew that the image was a dessert chart ( because an ship model concluded this ).

Are there more smartphone users than have devices?
How many more?
Do you know of any persons who don’t fall under either of these categories?
How many is that?

Setting aside the realities of large language model ( LLM) hallucinations—where a model just makes up plausible-sounding “facts” —for a moment, the opportunity to learn more about images and data in this way could be revolutionary for blind and low-vision folks as well as for people with various forms of color blindness, cognitive disabilities, and so on. It might also be helpful in education settings to assist those who can see these charts as they are able to comprehend the data contained therein.

What if you could request your website to make a complicated chart simpler? What if you demanded that the line graph be isolated into just one collection? What if you could request your computer to transform the colors of the various lines so that it works better for your type of color blindness? What if you asked it to switch shades in favor of habits? Given these resources ‘ chat-based interface and our existing ability to manipulate photos in today’s AI devices, that seems like a chance.

Imagine a specially designed unit that could extract the data from that chart and transfer it to another format. For example, perhaps it could turn that pie chart ( or better yet, a series of pie charts ) into more accessible ( and useful ) formats, like spreadsheets. That would be wonderful!

Matching techniques

When Safiya Umoja Noble chose to put her guide Algorithms of Oppression, she hit the nail on the head. Although her book focused on the methods that search engines can foster racism, I believe it to be extremely accurate to say that all laptop models have the potential to intensify issue, discrimination, and hatred. We all know that poorly written and maintained algorithms are incredibly harmful, whether it’s Twitter constantly showing you the most recent tweet from a drowsy billionaire, YouTube sending us into a q-hole, or Instagram warping our ideas of what natural bodies look like. Many of these are the result of a lack of diversity in the people who create and build them. When these platforms are built with inclusively baked in, however, there’s real potential for algorithm development to help people with disabilities.

Take Mentra, for example. They serve as a network of employment for people who are neurodivers. Based on more than 75 data points, they match job seekers with potential employers using an algorithm. On the job-seeker side of things, it considers each candidate’s strengths, their necessary and preferred workplace accommodations, environmental sensitivities, and so on. On the employer side, it considers each work environment, communication factors related to each job, and the like. Mentra made the decision to change the script when it came to typical employment websites because it was run by neurodivergent people. They lower the emotional and physical labor on the job-seeker side of things by recommending available candidates to companies who can then connect with job seekers they are interested in.

When more people with disabilities are involved in developing algorithms, this can lower the likelihood that these algorithms will harm their communities. That’s why diverse teams are so important.

Imagine if the social media company’s recommendation engine was tuned to prioritize follow recommendations for people who discussed topics similar to those that were important but who were not in your current sphere of influence in any significant way. For instance, if you were to follow a group of non-disabled white male academics who talk about AI, it might be advisable to follow those who are disabled, aren’t white, or aren’t men who also talk about AI. If you took its recommendations, perhaps you’d get a more holistic and nuanced understanding of what’s happening in the AI field. These same systems should also use their understanding of biases about particular communities—including, for instance, the disability community—to make sure that they aren’t recommending any of their users follow accounts that perpetuate biases against (or, worse, spewing hate toward ) those groups.

Other ways that AI can helps people with disabilities

If I weren’t attempting to combine this with other tasks, I’m sure I could go on and on, giving various examples of how AI could be used to assist people with disabilities, but I’m going to make this last section into a bit of a lightning round. In no particular order:

Voice recognition. Researchers are assisting people with disabilities in the collection of recordings of people with atypical speech, thanks to the assistance of the Speech Accessibility Project. As I type, they are actively recruiting people with Parkinson’s and related conditions, and they have plans to expand this to other conditions as the project progresses. More people with disabilities will be able to use voice assistants, dictation software, and voice-response services as a result of this research, which will lead to more inclusive data sets that enable them to use their computers and other devices more effectively and with just their voices.
Text transformation. The most recent generation of LLMs is quite capable of changing existing text without giving off hallucinations. This is incredibly empowering for those who have cognitive disabilities and who may benefit from text summaries, simplified versions, or even text that has been prepared for Bionic Reading.

the value of various teams and data

Our differences must be acknowledged as important. The intersections of the identities we exist in have an impact on our lived experiences. These lived experiences—with all their complexities ( and joys and pain ) —are valuable inputs to the software, services, and societies that we shape. Our differences must be reflected in the data we use to develop new models, and those who provide that valuable information must be compensated for doing so. Inclusive data sets produce stronger models that promote more justifiable outcomes.

Want a model that doesn’t demean or patronize or objectify people with disabilities? Make sure that you include information about disabilities that has been written by people with a variety of disabilities in the training data.

Want a model that doesn’t use ableist language? You might be able to use already-existing data sets to create a filter that can read and interpret ableist language before it is read. That being said, when it comes to sensitivity reading, AI models won’t be replacing human copy editors anytime soon.

Want a copilot for coding that provides recomprehensible recommendations after the jump? Train it on code that you know to be accessible.

I have no doubt that AI can and will harm people … today, tomorrow, and well into the future. But I also believe that we can acknowledge that and, with an eye towards accessibility ( and, more broadly, inclusion ), make thoughtful, considerate, and intentional changes in our approaches to AI that will reduce harm over time as well. Today, tomorrow, and well into the future.

Many thanks to Kartik Sawhney for helping me with the development of this piece, Ashley Bischoff for her invaluable editorial assistance, and, of course, Joe Dolson for the prompt.