Accessibility Lightning Talks: AI for Accessibility, Engagement, Audio, and Access

Lessons on broadening engagement in accessibility innovation

Raquel Ronzone of the Howe Innovation Center says the center is an ecosystem support organization situated within Perkins School for the Blind, the first and oldest School for the Blind in the country.

Their goal is to create a more accessible world by connecting the disability and innovation communities. A key part of this is broadening this community by showing where and how everyone, regardless of their role or their sector, has a role in advancing accessibility.

What does it mean to talk about a disability tech innovation ecosystem? Think about it as one centered around people with disabilities and their allies. And around them are all ecosystem stakeholders.

That includes startups, academia, researchers, investors, corporates, test beds, governments, and more. In short, the innovation center wants to ensure that better co-designed products make it to market. Products that meet the real-world accessibility needs of the disability community.

Building the disability tech innovation ecosystem

They do these three ways. First, they go beyond the accessibility space to reach technologists, the people who are building tech as well as informing and advising on tech policy. To attract this group, the center creates programs that solve entrepreneurs’ problems.

For example, the center partnered with Mass Challenge, one of the world’s largest startup accelerators. They featured a disability tech carve-out among their cohort of over 100 startups. Additionally, they offered human-centered design training to all 120 startups in the cohort, because accessibility is important to embed across sectors.

Second, they’re doing a project called Pilot with Perkins to understand their own innovation needs. They also figure out how to pair with corporations and startups working in that space. As an example, Perkins took stock of their 3D printing needs on campus. Where and how can 3D printing and other forms of tactile information unlock accessibility for their students? Then, they explore how to engage corporations and startups to staff those needs.

They’re also focused on the end-user community. Who is the community being served by this? Whether it’s the auditory, visual, mobility, or neurocognitive communities, or multiple communities. The technology used, whether it’s AI, robotics, or more. Additionally, they want to meet the tech community where they are, such as tech conferences and events.

They’ve had success in engaging local and online communities with the tech that they used, like AI. They found a local generative AI community Meetup with thousands of 500 members, many of whom anecdotally have never thought about accessibility and how AI can unlock accessibility for people with disabilities. The center found communities through grounding tech principles, like responsible tech or humane or ethical tech.

The first lesson learned in connecting with these three groups is to lean into being the lone wolf in accessibility. It means you’re reaching new audiences.

Getting the most out of the collaboration

They also want to reach cultural organizations because disability is part of the human experience. Cultural organizations offer a touchpoint, an opportunity, to engage people in this. They connect with cultural organizations by working with universities and schools on project-based collaborations.

For instance, they partnered with a consulting group at Tufts University. They took an existing discipline at the university, such as business and marketing, and applied a disability lens on top of it. What is the national landscape analysis for adaptive furniture? And after a semester, the university came back to Perkins with a specific number. And now they know what that need is and how to meet it.

They’ve also collaborated with a group of computer science students at Olin College of Engineering on accessible data presentation. The data is available in multiple formats. They’ve taken stock of disability tech companies by country and state in the US.

It’s opportunities like this that Perkins takes the layer of accessibility and disability experience, which they know well, and use it as an opportunity to engage with college students. The students might not have ever considered accessibility while bringing their talents to computer science. The second lesson learned is that, whatever the opportunity, the answer is accessibility.

Additionally, the center is connected with libraries, museums, and makerspaces. They search for participation opportunities on panels and submit disability-focused media for book talks and film club nights, and more.

Perkins has a partnership with the Museum of Science to inform their content and programming on disability. To that end, they’ve held two events. One focused on the disproportionate effect of climate change on people with disabilities, an opportunity to engage the STEM community, who might not have considered this before.

The other event focused on the intersection of democracy and disability, and how people with disabilities can and should be able to participate in democratic systems. It was an opportunity to engage tech policy people and government members.

Expanding the community engagement

Perkins’ value as a partner comes from solving other people’s problems and removing their friction points. And in doing so, they deliver on their goals while delivering on their partners’ goals. Thus, the third takeaway is to solve their needs first, then solve yours.

Howe accomplishes this by preparing others to do the pitch. Make the case to them and help them make their case to their higher-ups, collaborators, and colleagues. Equip them with your pitch decks, talking points, and briefs. And show the value of your work and collaboration through the data.

For example, in Howe’s partnership with the Museum of Science, the museum wanted to get more new attendees while Howe wanted more disability focused programming. Through the partnership, they can accomplish both. Another key lesson is to give more than you get when it comes to partnerships.

It’s also key to listen and respond. As a result of their broadening and deepening engagement with stakeholders across the ecosystem, they understand their pain points and can solve them.

As a result, the Howe Innovation Center is building a user testing pool to streamline testing among the disability and innovation community. They’re also developing a Slack community to connect all the global members of the disability tech ecosystem, creating a centralized place where they can share news, updates, job opportunities, job asks, mentorship offers, and more.

Audio engineering by people who are deaf and hard of hearing

Keita Ohshiro is a lecturer at the New Jersey Institute of Technology who is a PhD student focused on accessibility and technology. Keita is doing research on audio engineering by deaf and hard of hearing individuals.

How audio engineering works

Audio engineering involves recording, manipulating, and reproducing sound for music, podcasts, TV, radio, and so on. The typical workflow is recording is to capture the instruments and speech. Then, you import the audio tracks into your computer for editing. People use software called digital audio workstations (DAW) for audio engineering tasks.

In editing, you try to make multiple audio tracks sound good. The final goal is to produce the best-sounding artistic product. It’s not just a procedure. It’s more creative and artistic work. Two engineers can produce very different results from the exact same tracks.

Audio engineers also do tasks like balancing track volumes, timings, stereo placement, equalization, inserting sound effects, and so on.

Background on deaf and hard of hearing participants

How do deaf and hard of hearing individuals do audio engineering? To explore this, Keita interviewed 12 deaf and hard of hearing audio engineers. Eight participants work in music and four in podcasting.

Many of the participants describe persistent insecurities due to their hearing. As one participant said, “My ears can be a little deceptive.” They also said it took extra time and effort for them to do audio engineering. One participant became deaf during their career. They said, “If I didn’t have the hearing loss, it would have taken half the time at least. Because I could trust what I was hearing.”

In addition, some participants lack an innate understanding of certain sound concepts. For example, a participant born with a single-sided deafness said, “I can’t ever really experience stereo sound, the same way that people with two ears can.” What’s their workaround? One of the main workarounds is visual editing.

To better perceive sound, they leverage visuals within a DAW. For instance, looking at waveforms, you can tell that visually that someone is laughing because of a fishbone shape. Another waveform shows an explosive sound like a “P” or “T” because of the big spike at the start. In editing, you can trim it down, so the explosive sound doesn’t blast your listeners.

These visuals give deaf and hard of hearing audio engineers more confidence about what’s going on with the sound. It helps them save their hearing energy because they tend to get tired easily when listening. Of course, visuals can only take you so far. As one participant said, “I can depend on my eyes and the graphic readout only so much.”

And another participant said, “You do end up with a complicated guessing game.” Therefore, it’s not possible to see every nuance of sound. As a last resort, participants turn to their hearing friends. They ask peers to give feedback to ensure the quality of work. Most audio engineers do this. But deaf and hard of hearing audio engineers rely more on it.

That said, asking for help isn’t always easy. It can feel uncomfortable to keep asking for help, too often, too much. You might need someone who understands audio engineering better to get higher-quality feedback. Plus, sometimes you want to be independent, as a creator, without being influenced by others too much.

Audio engineering technology and future research

Regarding the use of technology, it turns out that deaf and hard of hearing audio engineers use almost the same technology as hearing audio engineers. Like DAWs and plugins.

One exception is the hearing devices, like hearing aids or cochlear implants. These can introduce their own set of frustrations. For example, most hearing aids are designed and adjusted for speech, rather than music, which contains a broader range of sound. They are often simply not good for music or audio engineering purposes. And finding an audiologist who is comfortable adjusting the settings for musical work is another challenge. It requires the right equipment in the lab and expertise.

Another thing is how they learn and grow as an audio engineer. Based on the research, I noticed a difference between the music and podcast engineers. All eight participants in music came through formal education, such as degrees in music technology.

The four podcasters learned audio engineering and podcasting by themselves through trial and error. They often developed their own ways of doing audio engineering. Sometimes not optimal, yet it somehow works for them.

They felt that it’s too much of a hassle to try out new tools or workflows as they spend so many hours learning and trial and error. That might limit the opportunities for them to expand their skills.

That’s where Keita’s results currently stand. The first task is to understand the current state of accessibility. And one of Keita’s future directions is to think about how to use AI for this. For example, adding sound design tasks in podcasting. Imagine you’re making a drama podcast, and you want to insert a footstep sound.

It seems simple, but you’ve got to think about a lot of factors, like sneakers or heels, wood floor or tile, and walking or running. Each of these factors changes the appropriate choice of sound. Those subtle cues can sometimes be tough for deaf and hard of hearing audio engineers to catch.

What if there’s an AI tool that can describe those subtle nuances of the sound, and help them select and insert the sound appropriate for the scenes?

Access in action: Promoting inclusion and creativity

Indira Ardolic describes Tech Kids Unlimited as a non-profit organization that provides educational opportunities for neurodivergent students ages 10 to 24. They started off doing tech workshops about animation, digital art, game design, and web development. At some point, they spotted a huge gap in the growing community in the lack of gaining or sustaining employment.

The organization has weekend and afterschool courses, weeklong summer workshops, an afterschool career pathway, and holiday programs. They expanded into work-based learning. The tech skill workshops have been a core part of their work. Their mission is to address the gaps in accessible experiences with technology. They want to teach computational thinking and tech skills in a nurturing and supportive individualized community.

They offer virtual and in-person programs. Their virtual programs bring in people from all over the United States. The in-person programs allow them to hang out and play video games that they bring together. They work out of the Ability Lab at NYU Tandon with social workers in each classroom. The program has a high counselor-to-student ratio, as well as financial aid and many free programs in Career Ladder.

The TKU model uses a combination of universal design for learning, social emotional learning (SEL), connected learning, design thinking, and project-based learning. This model allows them to enjoy the programs, gain new skills, and make friends.

Access is a practice, not a check box. It’s something that they must keep doing, finding ways of doing, and ways of applying it. It lives in every part of the ecosystem at Tech Kids, including curriculum, media, and mentorship. When everyone makes space for neurodiverse creativity, everyone benefits.

Using AI to build bookmarklets for accessibility testing

Damian Sian is a design accessibility program manager in the design organization within Adobe. Damian had an epiphany that he doesn’t rely on automated testing tools from companies that don’t do the work he does.

He conducts content accessibility training. He only focuses on the things the attendees can control. They use a content management system, so they are limited in what they can do. Damian is not a developer, engineer, or programmer. Yet, he created his own testing tool. In the past, when he had ideas of things to do, he had to rely on others.

Linking text problem explained

One of the things he talks about is linking text. A webpage has three cards, each with an image, text, and a button that says “Open.” A screen reader user tabbing through this interface would hear “open: Link,” “open: Link,” “open Link.” If they pull up the rotor with the voiceover tool, they’re going to hear “open, open, open.”

He tells students to expand on that by grabbing the adjacent heading and instead say, “Open: Text to image. Open: Generate video. Open: Text to vector.” And Damian builds this lesson, he thought to himself, why not build a tool that would help them understand how to do that in their situation? This way, they can do the lesson and feel confident in doing the work.

For the last part of the lesson, he teaches them that automated testing tools will only get you so far. And this scenario that Damian just shared wouldn’t get picked up by aXe or WAVE. ANDI and Arc ToolKit will give you a warning.

Building the accessibility bookmarklet with AI

Damian wants them to learn the lesson and not have the conversation again. So, he built something using Cursor AI. He had long conversations with Cursor AI to build a set of tools based on these parameters:

• Only look at <a> links inside <main>. Ignore the <footer> and <header> of the page.
• Rule one is if you have the same link text that’s going to different destinations, add an ARIA label to make them different.
• Rule two: If you do an ARIA label, it must begin with the visible, inner text. “Learn more about what? Read more about what? Free trial of what?” Otherwise, it violates SC 2.5.3 Label in Name.
• Rule three: If you put blue link text inside of a block of non-interactive text that’s black, it must have an underline. Links with a class of “action-area” are excluded from this.

Again, Damian isn’t a coder or a developer. Yet, he made a graphical user interface. It’s a bookmarklet that analyzes the page and gets granular about the environment that they are working in. It picks up CSS style names, component names, and that’s germane to this group that he’s training. The tool shows a screen reader text, it shows why it’s a failure, and it shows the suggestion. All of it was built with Cursor AI.

Can you give them the right answer? The right answer is: Pick up the H3 that’s right above the link. The tool will give them the confidence to do it on their own.

Incorporating AI into work

Why does it matter that Damian did that? He thinks that this is a conversation that many of us are about to have. If you go and ask your leadership for head count, they’re going to ask you what you’re doing with AI.

It’s time for everyone including non-developers who have ideas to get ahead of this and start having good stories to tell because it’s going to come up. If you’re not a developer, and you have ideas, don’t let it stop you from building your own tools that do the exact thing that you need them to do.

The cool icing on this cake is the development team for Adobe.com were making their own testing tool. And when they saw Damian’s testing tool, they asked if they could have it. Go out and use Cursor AI and any other tool that you can. Build your stuff. Don’t let anything stop you. And keep going at it until it works.

Equal Entry

Accessibility technology company that offers services including accessibility audits, training, and expert witness on cases related to digital accessibility.