AI for Accessibility: Opportunities and Challenges

All right. Hi, everyone. Thank you so much for inviting us here today. We thought we would like to talk about some of the opportunities and some of the challenges for AI, for accessibility.

So... Thank you very much for the nice introduction. Cindy, do you want to... Yeah.

This is Cindy. I'll just do a quick description. I'm a blind white woman with long hair that gets stuck in my name tag and the microphone. And that's good!

All right. And I'm Shari Trewin. I'm a middle aged white woman with lots of smile lines on my face. That kind of sums me up.

So our agenda, what we thought we would cover today, is a fairly kind of high level flyover of some of the main opportunities. Some of the main issues. So we'll start with: Why discuss AI and accessibility? Some of the applications that are already out there and upcoming.

And then the very important topic of bias in AI. And some of the challenges that we see for applying AI in accessibility. And then we'll end up with some recommendations, some takeaways and actions, that people could use to bring back to your own organizations or your own work. And some resources.

And I forgot to say there was a QR code on the first slide. We have a real big version of it at the end. So you can grab a picture of that. And that will take you to a page with a list of all the resources and links that are on the slides here today, so you can follow up.

Okay. So... Why discuss AI and accessibility? I wanted to start with a story about captions.

In 2020, I was one of the accessibility chairs of a large virtual conference. And we had a dedicated human captioner. Live transcribing the sessions. In a separate live feed.

So on this slide here, I'm showing a simplified version of what an attendee using the live transcript might see. During a conference presentation. So there's a slide from a presentation. With a transcription window to the side.

During the conference, I spoke to a hard of hearing attendee who uses captions to supplement what he can hear. And he told me that the live feed was having quite a delay for him. So he also used the automated captions that were streamed through the conference provider. Let's add them to the view there.

These had less of a delay. But they had accuracy problems. Especially for people with foreign accents or atypical speech. So...

He also turned on the automated captions in his browser. Which used a different speech-to-text engine. I've added those to the screen too. And...

He supplemented that with an app on his phone that used a third different speech recognition engine. And that was capturing the audio that was playing out from his computer and transcribing it for him. So that's four sources of captions for him to read. Not to mention looking at the slides and the presenter as well.

Now, ideally, the web conference tool would have provided better support for the live captioner in the first place. But in the situation that he was in, the AI-powered captions were helping him to access the conference. But it wasn't a very usable experience. He had a huge burden to manage his own accessibility.

And I'm telling this story, because I think it nicely illustrates some of today's challenges and the opportunities for AI and accessibility. So on the opportunity side, we can see that AI can be very empowering. Can support people in independently accessing content and the world. On the challenges side, we also see the AI was exclusionary.

It didn't convey the words of people with atypical speech. As well as it did for other people. And in general, we know that AI can be biased. And we'll share more examples of that as we go through.

And then finally, AI does make mistakes. And that's a really important factor, when thinking about AI as an accessibility tool. So I want to review some of the specific opportunities. Just fairly quickly.

Highlight at the beginning just some of the ways that AI is already being used or explored in accessibility. I'm gonna focus on apps and digital access. Because we're from Google. So speech-to-text has many different applications.

Right? It can generate captions for video authors to edit. Or for consumers to use. It can enable voice control for devices.

Like smart home controllers or personal assistants. And it enables voice dictation when typing is hard. And as speech recognition improves, it's becoming a mainstream way to control digital devices. And it's essential that people with atypical speech aren't excluded from that.

So researchers at Google and Project Euphonia investigated the potential for personalized speech recognition engines for people with atypical speech. After they recorded a few hundred utterances from hundreds of people, over a million things overall, and built a personal model for each of 432 people, and they found that the personalized models really could significantly reduce word error rates for the speech recognition. So they could reduce it from 31% down to 4.6% in their research.

So it's not just a significant improvement. But actually a really meaningful improvement. Which brings it to that point where it might actually be usable and useful and beneficial. And in fact, sometimes for some people those personalized models outperformed a human transcriber who wasn't familiar with that person and their speech.

So these personalized speech models look really promising. And in future, maybe similar approaches to this could be taken to build personalized models for gestures or for typing or touch screen interactions. Things like that. Okay.

I want to play an example. Hopefully this will come through everywhere it needs to come through.

This is an example of Dmitri Kanevsky. He's a researcher and an inventor. And he's using his personal speech recognition model. So here goes.

Let's see if we can play this.

[sound is muffled] Okay. So the transcription is... Make all voice interactive devices be able to understand any person "speak" to them. So it was not perfect.

He said "any person who speaks to them". But it was pretty close, after he was using his own personal model. So this app that Dmitri is using is called Project Relate, that anybody can use to build their own personalized speech model. In February last year, at this meetup, Pan-Pan Jiang from Google gave a longer talk, specifically about Project Relate.

So go back and check that out, if you want more detail about how they do machine learning for speech recognition.

Okay. Another form of audio is environmental sounds. So Dhruv Jain and colleagues at the University of Washington and at Google developed an app called ProtoSound. That identifies sounds in the environment.

And then displays the name associated with what it currently hears. Like in the example that I'm showing on this slide is a picture of hands playing on a piano. And the app screen is showing "piano", and the decibels of the sound. So it allows its users to make just a few recordings of sounds that are personal to their environment, and then train a custom AI model.

And it also has a library of pretrained sounds like a fire engine. Something that would be hard for you to kind of track down and record yourself.

It could be a project!

I just remembered one more thing. Just a little update on the Project Relate. Since last year's presentation, there has been a speech accessibility project launched by UIUC. And it's a partnership between academics, tech companies, and organizations representing users.

With the goal of gathering a large corpus of atypical speech data that's not just available within Google, like the previous project data was. But actually more broadly available to the academic community, to further progress research in speech recognition. So I just wanted to mention that. Okay.

Moving on from speech-to-text, I think we probably all know AI models can also identify objects in images. So Google has a Lookout app. Microsoft has SeeingAI app, and these apps are like a visual assistant that providers a kind of talking camera that describes what it sees in real world scenes, as well as on screen contents. So within the app, there are specialized different AI models for different kinds of tasks, like finding text, or identifying currency, or reading bar codes.

Those sorts of things. And to do this, they need crisp and pretty well framed images. Lookout app provides audio feedback to help users that are blind to take good images, and then adds in image processing to improve the sharpness of the images as well. It also uses AI models that are trained with images that were taken by blind people.

So they are a better fit for the kind of tasks that it needs to carry out. And just like with speech-to-text, it's also valuable to personalize these kinds of models. Because they can only really recognize objects they've been trained to recognize. So researchers such as Kyungjun Lee and colleagues at University of Maryland, College Park, they're working on methods for people to train their own object recognizers, to identify things that are important to them in their home.

Okay. Now let's think about reading. Many people find reading a struggle. Whether that's because of dyslexia or low vision or lack of fluency in the language.

Android reading mode is an app that makes reading easier by providing you a personalized view of the text you want to read, and then reading aloud that text with highlighting. And it takes advantage of some of the newer speech-to-text models that have more and more expressive, more natural voices that are easier to understand. And it also uses AI technology to identify what should be read out. If you're pointing it at a web page, there's a lot of things on a web page that are navigation, and pieces that are not the main content.

So it can also help to reduce clutter, as you read.

And finally, looking a little bit further ahead, large language models like ChatGPT have been in the news a lot lately, impressing people with how they can generate text, rephrase, summarize text. There was a Google research study published last year that had specifically looked at their potential for use with human-AI writing tools. Working with people with dyslexia, they explored a prototype kind of email authoring assistant. It could generate outlines and suggest rewrites.

They came to the conclusion in that paper that the state of the art is not quite ready yet to provide reliable writing support. But there was a lot of interest and potential there too.

So... We've looked at just a few of the applications. There's obviously many more. Like robot assistants and self-driving vehicles.

There's a lot of exciting opportunities. But before we get too carried away, you probably know that AI models can have harmful biases. So I'm gonna hand it over to Cindy now, to introduce the AI bias topic, and how that might surface. Go for it.

Thanks, Shari, for all of those examples. So now I'm gonna talk about negative things.

[laughter] So... But first I would like to briefly go over the machine learning development pipeline, and this is a very general overview, targeted toward an audience that does not know machine learning. And each developer and company will have a different process. But kind of the point we want to make is there's different phases, and there's opportunities to introduce and mitigate bias at these touch points.

So the first phase is dataset development. So machine learning models must have data to learn. So identifying and developing datasets is extremely important. These datasets help determine what the model can and cannot do, and as you can imagine, what the model can and cannot do can introduce bias.

Often developers identify existing datasets that fulfill their needs, and then they may identify gaps and curate new datasets to fill those gaps. In the next phase is model development. And in this phase, developers train the model with data from the previous phase, and then they choose different techniques for how the model will learn, and I won't go into much detail, but as you can imagine, these decisions that are made about how to design the model can introduce bias.

And the next phase is evaluation. And mitigation. And so this is where developers and researchers and analysts evaluate whether the model does what they wanted it to do. And create mitigations.

And what's important here is: Benefits and harms can only be measured if they're in the evaluation. So whatever is not evaluated is not gonna be measured or mitigated. So finally, the model is launched, and there's ongoing assessment, impacts, and this is the point where we learn about benefits and harms that were not anticipated. And so it's important to kind of stick with this development cycle for maintenance, in order to address any unanticipated harms.

So in the next slide, I'll talk about just a few types of AI bias at a higher level. These examples kind of all relate to accessibility and disability in some ways, and touching on some intersectional concerns. And I encourage you to consider this just as a starting point. And again, to look at our resources that we'll give at the end.

The examples I'll go over concern: AI relies on averages, alt text, misclassification, and data may exclude disability. So AI relies on averages. This shows distribution with clusters and averages. Recall that data development phase I mentioned.

Many datasets reflect the most common, not the most representative data. And so people with disabilities or accessibility needs who are often considered societally to be at the margins just aren't going to be in the datasets that are most easily available. And so just the kind of... The way that machine learning works in relying on these averages can have negative implications for people outside of that.

Including people with disabilities. Go to the next slide. So next, AI may misclassify information. And this as an impact for accessibility use cases.

Like automatically generated alt text or image descriptions. So this can happen in a couple of different ways. So as one example, groups of pixels on an image may not be labeled representatively in training data. So as an example, some bias that I've come across is women wearing professional uniforms, like firefighter uniforms or military uniforms, that have been stereotyped as only associated with men -- there might become a learned association in that model between uniform and man.

And that's not accurate. And so that is one way bias can happen. Another way this can happen is when certain labels are just not included in the data. So as an example, if you take the image of Leyla on this slide, who is from the Disabled and Here stock collection, this collection is great, because each photographee wrote an information about how they identify.

So we know that Leyla identifies as Black, nonbinary, and disabled. Yet most model training does not have gender labels, except for binary gender labels. And so in this case, it's not a matter of labeled with the wrong thing. It's a matter of not even being included in that dataset.

And that's the same case for disability, as well. So you might think of some solutions -- might involve working on the training and trying to mitigate those problematic learned associations. But we also have to ask, you know, the potential for harm in getting some of these classifications wrong. Like gender and race and disability.

People have very different ways of identifying. And so we have to ask whether these types of labels are even appropriate in applications like alt text. Next slide. The last form of bias I'll expand on today is: Missing data.

There can be missing data, such as disability. And this can have downstream impacts as a result. So in this image, you'll see Emily Ackerman, who is a wheelchair user. And she encountered a delivery robot in the curb cut.

She was crossing the street and really needed the robot to move. Whereas people who are walking could choose to not use the curb cut, Emily relied on it. And this was a pretty dangerous situation. She was crossing at a traffic light, which was not going to stay green forever.

So while this robot is proprietary, and we don't know exactly how it works, it demonstrates that, you know, if AI does not recognize some people, it can respond in inappropriately and potentially unsafe ways. So there can be kind of downstream impacts. Unfortunately, delivery robots could actually be quite useful for disabled people. So a lot of these use cases are...

There's potential for real benefit. But they need to be able to recognize and respond to diverse people, things, and public infrastructure. I would encourage you to read more about Emily's experience. We have a link in the resources doc.

Unfortunately, some of her online activism did result in cyberbullying. So there's still a lot of room to kind of do better about the advocacy around trying to prevent these dangerous situations. Next slide. So next I'll talk about challenges accessibility researchers have encountered when attempting to prevent or mitigate these biases.

So on the next slide, they are performance inconsistencies may interfere with access, acquiring inclusive data, and accessible verification challenges. Next slide. Performance inconsistencies may interfere with access. Trade-offs may be made between immediate application and accuracy, and since outputs from common machine learning applications are probabilistic, meaning it's making a guess based on its training and learned associations, outputs can be unpredictable.

And there are implications for this in accessibility apps, kind of like... If you remember the Lookout example. So I'm gonna read a quote from Laura Smith. She is blind, and wrote an article of a day in the life of using technology.

She wrote about a task where she used an AI tool. She said: My daughter wants to wear her yellow tights today. My partner has now left the house, and the fabric on each pair feels the same, not giving me any clues as to what color they are. I use a color identification app, and again take a photo of the clothing, before the technology lets me know its color.

Orange, the app replies. My daughter only has yellow, blue, or white tights. So I guess that orange probably means yellow, and I get her dressed. So Laura had to use prior knowledge to deduce that orange meant yellow.

And it's great that she has access to this color recognition tool now, but this creates a different type of labor for her that often goes unrecognized in technology design. So focusing on quality improvements for specific use cases asks developers to not rely on average successes. And this is sometimes a difficult argument to make, particularly when kind of repurposing models from mainstream applications. Because generalizability is so often a goal of how can we use this model for as many use cases as possible.

So some of the personalized models that Shari mentioned are really, really promising in this regard. But they're not yet widely available, and there still is kind of a predominant focus in mainstream machine learning that average working okay for more use cases is better than working really well. And this shows some of the downstream impacts of that, that are challenging to get around. Next slide.

So next I'll share some challenges related to collecting inclusive data. This is actually pretty tough, even when people want to. So finding enough data is difficult, as often people are underrepresented, are low incidence populations, and they also may be underrepresented in digital archives. So maybe they're populations of people that don't have as much data online or digital traces.

So there has to be a lot of intentional effort. Also, disclosing personal information, like disability, can lead to negative consequences. As there is an Americans with Disabilities Act for a reason. Because of historic and ongoing discrimination.

So building in privacy protections and trust is super important. Next, people with disabilities are a heterogenous group. And using inconsistent terms to describe their experiences. So it's pretty hard to know, even if data is labeled in a certain way, whether it qualifies for the dataset you're curating.

So, for example, some people use the term "disability". Some people don't. Some people who use the same disability terms may have different -- present in different ways. And so this makes it tougher to just rely on labels.

And also synthetic datasets, which is actually pretty common in health care applications, can lead to disability stereotyping. So I'll give an example regarding augmented communicators, who use technology like switches and eye gaze to type. And this process is often very slow. So there's been a lot of effort to improve typing accuracy for these users.

So a study a while ago asked crowd workers in an attempt to generate a dataset, asking them to type sentences that they thought augmented communicators would type. And unfortunately, they tended to type very medicalized statements. Like... I need a ride to the doctor.

Will someone bring me some soup. And this overrepresented an association between medical needs and people with disabilities that doesn't necessarily represent what they would want to talk about every day. But as I mentioned, sometimes collecting authentic data is very fatiguing. Asking an eye gaze user to come into your lab and type 100 sentences -- that might actually be unethical.

So there's kind of a tension here, in how to ensure that those data collections are authentic, when they could be very fatiguing. Next slide. Finally, I'll talk about accessible verification challenges. So one way to mitigate or -- maybe not mitigate bias, but kind of create expectations around it -- is to give users the tools to understand why they're getting certain machine learning output.

However, some applications for AI, for accessibility, remediate fundamentally inaccessible information. So, for example, going back to alt text, it provides access to images that blind people may not be able to see or see enough to get a sense of whether the automatic alt text they are reading should be taken seriously or not. This topic is very underresearched. I would really love to do more work in this area.

But in 2016, I collaborated on a project, where we evaluated how blind people trusted automatic alt text, and they tended to overtrust it. So, for example, we showed this image of Hillary Clinton on stage to a participant, and it was part of a tweet. So the tweet said: Some on the other side say that our best days are behind us. Let's prove them wrong.

The AI generated image description read: I'm not really confident, but I think it's a man doing a trick on a skateboard at night. [laughter] Even though the confidence rating was low, this participant said: I probably just would have retweeted it, thinking it was a photo of a skateboarder, and I would have been in so much trouble with my friends. People would have been like: I thought we don't get political! So this example is extreme.

But you can imagine -- I didn't make it up. It really happened. But that's a pretty extreme example. And you can imagine that more subtle mistakes may be harder to capture and convey uncertainty about.

And often the potential consequences of an incorrect description are determined by the context. So in this case, Hillary Clinton is a politician. At that time, she was running for President. And this particular person did not want to come off as being too political.

And these factors that can affect the consequences are pretty undeterminable by AI. So creating systems that can encourage healthy skepticism when AI is remediating access to fundamentally inaccessible material is a huge open question that I think is a challenge. But definitely something worth pursuing, so that people can have the right expectations and the right amount of trust in AI systems. And I will pass it back to Shari now.

Okay. So what can you take away from this whirlwind tour? So we have four takeaways to go over here. If your organization is using AI in an accessibility or other context, or you have some involvement or influence, here are some things that you can do to help maximize the benefit and address some of the challenges that Cindy outlined.

So we'll talk about four steps. Advocating for inclusion, identifying risks, partnering with communities, and setting expectations. So going back to that development pipeline, there are so many potential sources of bias and different ways to use AI. It can be hard to know where to start.

Whether you're an advocate or an accessibility specialist or an AI engineer.

And there's opportunities at each stage of the development pipeline to be more inclusive. So a really powerful thing to do is ask good questions. And this works whether you're developing the models, you're adopting existing models, you're thinking of using a third party product that is driven by AI in your organization, so if it's dataset development, we can ask the questions about the diversity of the data. We can push for more diversity.

If it's model development, we can ask: How is this data being translated into input for the model? And how is the performance of the model being measured? And then in the model evaluation, how well does that model meet accessibility use cases -- might be a question that nobody else can asked so far. And then when the new application is launched, we can advocate for feedback channels with people with disabilities.

So a really useful exercise right at the start of any project is to try and identify the risks. Who might be impacted by this specific proposed application? So I'm gonna share a kind of question asking exercise that you can do to help identify the risks for a specific project. And this is based on something called the Fairness Action Framework I presented in 2021 in a workshop in the Fact Conference with Philip Jenkins and Erich Manser from IBM.

But we tailored it a little bit here to assistive applications. So the first set of questions help to explore what's at stake. And the first question to ask is: Does this application use human data? This means quite a lot of things.

Quite broad. Data about humans. Data containing humans. Data generated by humans.

All count. So image understanding in the real world will have people in the images. So yes. The delivery robot operates in the world, among people.

So yes.

And then a related helpful question is: Does it use human input to control something? So, for example, voice control of a smart speaker. There is another. You have more yeses.

You have more risk. And then finally: Is the AI output used for making decisions that impact people? And how much do they impact the people? So usually when you're using AI for accessibility, the answer is yes.

But there's varying levels of impact. It's using tights that are green instead of yellow -- has social consequences. Identifying currency wrong has financial consequences. And not moving out of the way of the person who needs to use the curb cut can have fatal consequences, even.

So it's helpful to take a bit of time to really think through what's at stake for the humans in the data and the people using the tool. So this would give you a kind of a high level overview of the potential consequences of mistakes or bias. So now let's dig a little bit deeper with a second set of questions about who is at risk. So some of the questions here would be: How might differences between people affect the data that's being used?

Which groups of people would have input that looks different? Or might not be able to provide that type of input at all? And this isn't limited to disability. There's all sorts of key stakeholder groups, as the examples that Cindy gave showed.

Okay. Another question: Who would be adversely affected by this application? It could be the user, when the AI does something wrong. Or it could be the bystander, the person in the street.

And who is the one who actually bears the cost of the errors? If it's not the organization that's using the AI, then there's a risk that they won't consider the cost that the other users are bearing.

So again, we can think quite broadly about these adverse effects, and the cost of mistakes.

Another really important question, and a potential for a red flag here: Is this a domain where there's historical discrimination? And are we relying on historical data to build the AI? Because that should lead to some further questions. Can this project improve on the past?

And how would we go about doing that?

So yeah. What specific outcomes should we be looking for? And how and when could we check on that, as the project goes forward? So thinking about these questions can help you identify which disability groups, which other groups are most likely to be impacted by the proposed application.

And these are your key stakeholders.

So then you can ask: How well are those key stakeholders represented in the data that's available? And you can try to diversify the data. You can try to explore personalized solutions. But if we don't have any data, we can't even tell if there's going to be a problem.

And then finally: It helps to ask questions about what success should look like. So what does it mean for this application to be fair, to be treating people fairly? There's a lot of different ways to measure fairness. And it's important to think up front about what definition makes sense for your particular project.

So there are ways to measure fairness at a group level, where you're statistically comparing people in an in-group and an out-group, there's individual fairness metrics that mean that one person with a disability should be treated like another similar person who doesn't have that particular disability.

So I think defining these fairness goals up front will also help you figure out which of the key communities to work with, to be able to test if you've achieved fairness.

And then another red flag to watch out for is where you have a feedback loop. Where the decisions made by the AI create the new data that's fed back into the AI. So that if it started with a bias, it perpetuates and even strengthens that bias as it goes. So that's also a really important thing to check for.

Then we can ask how well does it work for our particular stakeholder groups. Is there an alternative, when we can't apply AI?

For example, if there's a fingerprint scanner to get into the building, what's our plan for somebody who doesn't have fingers?

And then what happens when the AI makes a mistake? Will the user know that the AI made a mistake?

And then one final question is: Is this AI being applied in a way that people can challenge? Or override? Or get explanations for, for the AI decisions? So not all of these might make sense for every project.

But thinking through them, you can sort of get a good idea of what stakeholders you might need to work with. So... Speaking of stakeholders, I'm gonna pass back to Cindy to continue with community-based approaches. Yep, almost done.

I love that fairness action framework that Shari and colleagues came up with. So next, I'm sure you often tell other people to do it, but we recommend partnering with communities. And I actually think doing things like the fairness action framework with communities could even be really helpful too. One thing that I think could potentially be a little bit different is incorporating AI education into your community engagements, as a way to give back to communities.

And also just in my experience, I do a lot of human subjects user research with people with disabilities. And I hear a lot that AI is mysterious. And they will just never understand. And that doesn't have to be true.

And I notice that feedback really changes and becomes, in my opinion, more rich when you have even very basic modules on: Here is the development cycle. Here is how things work. And I think that would be a very valuable thing to incorporate into community-based perspectives, that I imagine many of you are already advocating for in your organizations. And next slide.

We also recommend making reasonable... Setting reasonable expectations about AI's capabilities and limitations. At several touch points. I know this can be really tough, when we work in organizations that want to promote products in a really positive way.

But really trying to advocate for some of this transparency and explainability, so you're having explainable interfaces with some of the things that Shari mentioned, like getting an explanation, getting clues as to whether the accuracy... Conveying uncertainty, incorporating this type of information about how use cases that it works really well on are not in user manuals. And kind of doing measured storytelling and promotion, so people understand that AI is not omnipotent. There are use cases where it might be helpful or not.

And on the next slide, we've listed out just some of the resources from -- these mostly show from Google, but on our list, there are many, many more. So on our next slide, thank you for coming today. Feel free to tag us on Twitter, if you have something cool to say, or feedback to give. My handle is @clb5590.

And Shari's is @sharitrewin.

We also have a QR code to get the resources. And we have a tinyURL, which I'll recite, before... We have a giant QR code on the next slide. But the tinyURL is: TinyURL.com/a11ynycai.

And it's a view-only access Google Doc. So I'll let Shari advance to the QR code. And we look forward to taking any questions.

This question comes from an attendee on the YouTube livestream. And it is: How do you measure variance across race, age, gender, and other protected classes? And what would be an acceptable level of variance?

I think for me, there's no one answer to that question. And that's part of the set of things that should be considered at the beginning of a project for this particular project, and these kind of users. What is an acceptable level for us, and how are we going to measure it. So...

There are a lot of techniques for statistically measuring fairness and variance levels. But as Cindy was explaining, the data doesn't necessarily always contain information that you could really use to assess. Whether you've got fairness or... Along any of those lines.

So I think individual fairness measures are really interesting. And that sort of shifts the challenge a little bit from trying to have all the information in your data that you need, to trying to define what the model is really trying to achieve, and then coming up with test examples that you can probe, whether similar people that should get a similar result actually are getting a similar result. Cindy, did you want to add anything?

Yeah. I mean... I think... So what I'm hearing in the term "protected classes" is sometimes that data is not available.

And I am aware of the attempt to use proxies of, like, oh, if race is not disclosed, what data is disclosed, that we can use as a proxy. And I would really caution that that can itself introduce bias. That may not have been there. I really...

I'm a big advocate for -- this is where qualitative research and community involvement comes in. Obviously a small number of people cannot represent the whole. But you can get a lot of clues of things to look out for, by triangulating your big data efforts with qualitative data, and I just have a lot of concerns about using proxies to approximate for protected class data. I think that might even introduce additional bias.

Hi. This is Craig. Thank you for that presentation. I wonder if you've seen any particularly good examples of sensitively done community involvement.

I think, Cindy, you shared an example about how it would be unethical to ask somebody who types with their eyes to come in and spend hours typing with their eyes, so that you can develop a corpus of data. Or... I know I work a lot with the Deaf and hard of hearing population, and asking them to come in and speak, when they've endured years of speech therapy, and have been made to feel bad about their voices...

That could be triggering or traumatic. Have you seen any examples of good community involvement? And are there any techniques that we could learn from? Where people have been able to reach out to communities, who aren't represented in datasets, and effectively partner with them to get that data?

Yeah. Thanks for the question. Because it's definitely not: Oh, go talk to people. Because that's a way to cause harm sometimes.

So things that I think help are to kind of first gain kind of a baseline understanding. Kind of doing some education. So I tell a lot of people who want to meet with me about accessibility... I'm like...

Have you checked out these YouTube videos? Or... So there's that piece. I also think demonstrating that you're bringing people in early, so doing things like the fairness action framework, and demonstrating like...

You can actually have an impact on the decisions that are made, not like... We already made all the decisions. And then we're gonna do... And I understand that sometimes that's all you can do.

But I have found it to be very effective, when you bring people in early and demonstrate that they can have an impact on decisions. I found that partnering with organizations that are already trusted in communities -- you can have kind of different forms of involvement. So sometimes an organization that's already trusted in the community, they can do things like help you vet your materials, and plan procedures, to say: Oh, this may or may not work.

This particular activity may not be accessible. So kind of... More like accessibility partners. And then maybe people -- because they have trust in that organization -- may be more willing and interested -- and I do think at some point making sure to compensate people, you know, I think in some ways just being transparent.

This is a transaction of knowledge, and we're gonna compensate you for that. Sometimes that transparency can go a long ways, so people know what they're getting into and what they're getting out of it. So those are just some ideas. I've also found...

When I have scaffolded conversations with people with disabilities about AI bias, that's been really appreciated. So again, as I mentioned, maybe incorporating modules on machine learning education or doing something, giving back to the community, asking people what they need, and organizations what they need. So I have had colleagues who donated five hours a week to 3D print whatever this teacher of blind students needed. And that was extremely valuable to her in her context.

Because she didn't have the time or the skills. I've heard of people going in and, like, you know... Reading materials or helping people make some stuff accessible, so... You never know.

But being open to the ways that -- what would be meaningful, as an exchange for the knowledge. I'll stop there. Because I'm rambling. But...

There is not -- I mean, I don't know that I could point to, like, this one thing. Those are just some things that I've seen have positive outcomes in the past.

Okay. So if a non-decorative image contains important and lengthy text, is it acceptable for the alt text to summarize the text in the image, to, say, 250 characters? The comment after. This is a sophistication dilemma.

My answer as a blind person, no. First, text should be in text. Not in an image. But I think if it is in an image, I personally think a summary...

Thinking about... I have a lot of thoughts on alt text.

But maybe there's a framing that's like... Oh, I'm doing the user a favor by summarizing. But there's another framing that's saying: I am deciding what that user gets to know. And is that a power play that you should make?

I think in the case of text, you can just type it out. Maybe in the case of very complex visual imagery you have to think a little more critically about that decision. That is my personal opinion. I would be happy to share more about that too.

I think there's sometimes technical limitations, as to how much text you can fit in, 'til you have to move to a longer description, that's somewhere else in the page. So... Maybe if it's something long, it would be better to have the whole thing... Or a summary and a pointer to the whole thing.

Or a pointer to the whole thing and a summary... Whatever Cindy says. So Shari makes a really good point. There are platform limitations.

Which -- I don't quite understand the rationale for. So sometimes what I do in academic publications, where there's an image of a graph that is very complex, I might share the main points of the graph and then say: See this appendix at the end for the full long description. And even better if you can link it from the caption to, like, give someone the ability to link right to it. So I hear that.

But... Just remember when you're writing alt text, you're not just necessarily doing a favor of summarizing information. You're actively making a choice about what the user gets to know and not know. And I hope that that adds a little gravity to...

Not to scare you. But to think about it a little more.

I have a question. It's Cameron here. And I'm curious what you think about AI as infrastructure. Sort of like...

Raw...

Can we rely on something like a large language model to be a commodity? Commodity infrastructure? Like AWS -- you can just scale up compute power. Can we think about AI in the same way?

Or is there risk involved in just treating it like an off the shelf solution? Does that make sense to you? What I just asked? Yeah.

I think so. And we're beginning to see that kind of thing emerging now. I think. Where there are very large models that have been trained on huge amounts of data that are a base, and then you take that model and specialize it to a particular application, by adding a much smaller amount of training data to modify it.

And are there risks in doing that? You're bringing along with you all the bias that was in the original data. But trying to kind of override it. And I don't know if we have enough research yet -- especially in the accessibility domain -- to tell us how good that is.

With the personalized speech models, that's kind of what we were doing. And we have some evidence there that that was pretty effective at solving that problem. So I think there's cause to be optimistic that those kinds of approaches can be really helpful. Because it addresses the data problem as well.

So follow-up to that: Were those personalized speech models -- were they built on top of existing generic speech models? Or were they built from scratch? Yeah, they were starting from an existing base model. Oh, amazing.

Okay. Thank you.

I read an article recently, as the whole ChatGPT wave was happening... And I'll find who it was that wrote it, but... The title was something along the lines of: ChatGPT and the Enshittening of Knowledge. Which basically spoke to the feedback loop you acknowledged in your presentation.

Where you've got an error in the initial data and it then produces new content that gets fed back into the new model and things like that. And as we get to a place where things like ChatGPT or like... I'm blanking on the visual imagery...

DALL-E... Start creating this monsoon of new extrapolated content or generated content, that then becomes the basis for crowdsourced next generation of content... That becomes normalized to a mean, or becomes kind of... A source of truth that was based on a lie that becomes an amplified lie that becomes an amplified inaccuracy, to the point where that is now a reference that's cited.

Like... How... At the point where this becomes a globalized crowdsourced... Echo of an echo...

How do we prevent against that feedback loop being inevitable? What protections can we put in place here that prevent that ten years from now, from becoming an irreversible problem?

Yeah. That's a big problem. Yeah.

It's a big question. [laughter] There's definitely a risk of homogenizing language, biasing language towards the language that's already on the internet. Losing representation of people who aren't already there on the internet. And I think it will take active effort to try to diversify the data that's being used.

And to try to perhaps cull some of that echo effect. Right? To not use as input stuff that was output from the models in the first place. So perhaps there could be some automated methods for...

There's tools that try to identify: Was this AI-written text? And they're reasonably successful. So I imagine we could use some tools like that, as part of the solution. Active efforts to diversify the input, as another part of the solution.

Maybe personalization for different regions could also play a part in there. I have no idea how this would be enacted, but it makes me think about like... What do we do to decide what's right and wrong and what's true and not true now? And I would argue that our methods for doing that currently are not perfect, but...

You know, the things Shari mentioned about: Can users challenge or even give feedback? I hope humans are a part of that deliberation. And could maybe help... Right?

Democratically deciding what we agree with and don't. But I know that's very idealistic. But to say we have existing methods, like in academia it's peer review. In other communities, it's maybe something different.

And what can we learn from those methods?

This is Walei. What is ChatGPT? No, just kidding. [laughter] So obviously AI is very prevalent right now.

And folks are trying to use it to solve everything. And I'm just wondering... Are there places that you think AI should not be going? Or is it just a matter of like...

It's inevitable, and we should just be more responsible about how we use AI?

I don't want to be quoted, but if you do look on Google, there are -- in the AI principles, there are a few domains that Google has decided it won't -- so I do think...

There are. And you know... Certain companies and organizations have kind of -- Drawn some lines. And I think the process of advocacy and the political processes in regulation play an important part.

Policies and regulations should play an important part in where AI is used and how AI is used too.

I guess the short answer is: Yeah, I guess there's applications where it's not appropriate. And we should be thinking about that more. Maybe. I wanted to ask a question about like...

How you assign the subgroup? Do you rate people with multiple disabilities, or have a lot less data on that, and they can't make models or do anything about that? How do you guys deal with that? In these applications?

You're talking about... When building AI models?

I think those are some examples -- remember Cindy's picture of people who might have been outliers in that data...

I can't speak for what all Google engineers do. But I think that an important thing to do is to try to think about those fairness metrics, and to use some of the qualitative methods, as Cindy is suggesting, to test. If you're creating an application that's making decisions that impact people or controlling something... Then in some situations, there needs to be an alternative way of controlling that same thing.

I think accessibility standards have an important role to play here. Because they kind of give a grounding in some of the basic things that need to be in every application, and it's just no different for an AI application. It still needs to be accessible. And so meeting accessibility guidelines actually gets you some part of that way as well, I think.

That's sort of separate from how the model behaves. Yeah. I was gonna say... I imagine a lot of you do the same advocacy in your organization.

Make sure it meets those existing standards. What I think I heard you talk about is: Multiple disabilities. And I agree. I think we could generally do better about that as an accessibility community.

Because so often, we break disability up into these very separated categories. I don't have the solution to that. But one thing I do encourage people -- and something I try to do in my own research -- is to not exclude people. And I usually end up with several participants who end up having multiple disabilities.

And I use that as an opportunity to learn, and not as an opportunity to be like... Oh, well, you can't participate, because you're not only this or only that. So even just asking more holistic questions about people's experiences, when you do have them in the room, will often reveal other facets and intersections of their experience that you didn't anticipate. That could be one way.

But I agree that as a field, we need to do better about that. Well, it's not necessarily specific to AI, but one of the things that our team is trying to do to break down those silos and try to think in a more holistic way is to work with groups that might be more outlier groups from the main group that we're thinking about. So instead of just trying to go to an organization that serves one particular group, maybe we will go and work with an organization that includes that.

But has a broader population. And that's something that we're doing in our team.

So this also comes from the stream. It's actually from Thomas Logan. And he says: I loved your real world reference of the mom trying to pick out the pants that were yellow instead of orange, as identified by the app. How would you improve that?

I don't want to say personalized models, because I've been saying that all evening. I think there are some things that applications can do to express their level of certainty in the language that they use. But as Cindy found in her previous research that she mentioned, people are still overtrusting the AI. So I don't think that's enough by itself.

But I think sometimes ChatGPT sounds very confident about everything that it tells you. And there's room for improvement there in using language to convey where there might be uncertainty. But that only really covers when the machine thinks that it's uncertain. Which doesn't cover all the errors.

This is the very accessibility advocate answer, but considering it also our job to work on removing structural inaccessibilities. So if those tights were accessible in some way out of the package, the AI wouldn't be necessary. And actually, Shari has done some workshops on exciting ways of thinking about: What if AI is not necessarily maybe focused on helping the human, but what if it could write more accessible code, or... Shari, what were you...

That was one area that I'm kind of interested in. Is that, as people are turning to AI to write code, we kind of have a call to action to make sure that we're asking: Is it writing accessible code? To try to make sure that we're not propagating today's inaccessibility problems in the same kind of echo chamber that you were mentioning earlier. Yeah.

There's potential opportunities to, like, turn the gaze, I guess I would call it... Inward. Onto... Could AI help modify structural inaccessibility?

And right now it's kind of remediating... Yeah. I think that if you can apply AI at the source, with the content creation, that's going to benefit more people in more ways. And there you still have the content creator there, who can identify any issues and fix them.

And so AI could save that person a lot of work. But reduce the risk of putting out something that is gonna be wrong. So I'm a big advocate for kind of putting AI as early in the pipeline as we can, rather than leaving it all to the end users. And that way, you'll reach more end users who don't have their own fancy AI tools as well.

We're gonna do one final question. Does anyone in the room want to...

All right. So last question is also from the audience. And by the way, that was a fantastic answer, I just want to say. How can everyday people, who are now using AI platforms in their day-to-day lives, ensure they utilize those systems with accessibility and potential bias in mind?

So people who are using them for accessibility? For, let's say, creating content.

To make sure that their own use filters out any kind of bias that might exist in the platforms. Yeah, I think we need tools. I think we need a lot more and better tools to help people do that. Because not everybody really knows about accessibility or understands what's good and what's not good about an alt text.

Right? I'm sure people have seen a lot of well-intentioned but unhelpful alt text. And descriptions of things. So I think...

We need tools, and maybe we need a different kind of accessibility education that focuses on that new task that people now have, of... Oh, I've got my captions now. Bang. Now we need to educate people to go and check their captions.

And what is an efficient way of doing that? And do we have good tools for efficiently remediating it? So I think it sort of changes the task a little bit for people. And we might have a little bit of work to do to catch up with that.