This article is based on a talk Kevin Berg, VP of solution engineering at Evinced, gave at A11yNYC .
Artificial intelligence (AI) can help solve the digital accessibility problem. In fact, GitHub did a controlled study on the impact of using an AI tool like GitHub Co-pilot can have on productivity and satisfaction when it comes to developers. In doing the study, they split 100 developers in half. Half used GitHub Co-Pilot to write a web server. The other half didn’t.
The study’s researchers expected AI would help the work move faster. But not to the point they found. The average time with Co-Pilot was 71 minutes. The average time without was 161 minutes. Being able to increase that amount of speed is a huge productivity booster.
If you dig into the study, you’ll find the developers enjoyed using Co-Pilot. That’s because AI easily took care of the more redundant parts, the more boring parts. It left interesting challenges for the developers to solve. In short, AI allows developers to save their brain power and mental energy for the problems they like to solve and remove the tedious parts.
The problem is that there are many different areas that need solving to remove the bias in AI. For example. Someone entered a prompt in ChatGPT to write a Python function to see if someone would be a good scientist. They used a JSON description of their race and gender. The return was biased toward white males, which is a huge problem.
Fortunately, this bias is fixable. Bias is one of the main problems with AI models. It has a significant impact especially where accessibility is concerned.
Another issue is AI not accurately responding to a user’s prompt. For example, someone provides a prompt of “Write a sentence that ends with the letter s.” ChatGPT presents a response that says, “The birds in the trees sing sweetly at dawn.” The sentence uses the letter s quite a bit, but it doesn’t end with the letter s.
People must be careful about their expectations around what kind of responses they’re going to get. In this case, they must consider what the prompt is, and how to communicate with these types of models to get the answers that they want.
3 Ways to Improve Generative AI for Accessibility
Evinced wants to use AI to to reduce the amount of tedium and increase satisfaction in working on accessibility issues. The goal is to do that in three different ways. First is intent detection. One of the great things that people can do with AI is use it to understand the context of the code that they’re developing or the type of design that they’re creating.
The second thing is to dig into these AI models with the expertise and understand how to handle prompt engineering. When it comes to creating prompts and trying to get responses that are both deterministic and correct, these prompts end up being extremely complex and often very long to get the types of answers people need to make productive changes.
The third is guardrails. We need to make sure to constrain the results from the AI and that they are accessible.
With these three areas — intent detection, expertise, and guardrails — people can use these tools in a way that’s helpful. Start with a design. How can people use AI to improve the process of design in general, and make sure that designers are always designing and creating designs that are accessible from the very start?
Design Assistant and Code Assistant Demos
How is it possible to ensure that AI models and tools like GitHub Co-Pilot return code that makes people’s jobs easier and faster as well as accessible that can be used by anybody? How can AI be used to find existing accessibility problems? Once found, how can it be used to fix them in a meaningful way? Evinced has developed a new set of tools to do that.
Evinced took the underlying data from within Figma, applied the visual aspects, and used both AI and machine learning models to determine what type of design was being created. Then, it mapped it to an ARIA pattern and provided the designer with guardrails that helped them to create accessible designs every time they were developing a new component.
Find Accessibility Issues Faster
How can developers find more accessibility issues when looking through the code that’s already developed? How can developers do it in a way that gives them the ability to reduce the amount of manual effort involved in going through every page of a website with a screen reader and a keyboard?
Modern technology makes it possible to find a lot more than any other tool. Like with Co-Pilot, it makes the developer’s job easier and less tedious. They’re finding the harder problems that are better to find and fix quickly. The next demo shows how the technology is used to detect more issues.
Finding and fixing problems demo
After identifying an issue, a developer can get a fix suggestion from AI. The developer would send the existing code snippet along with any prompts to AI and receive a new code snippet.
In the example, the suggestion for fixing a problem is to change an interactive element from a <div> to a <button>. At this point, the developer returns to the source code and makes the change to create a button and resolve the issue.
Video Highlights
- How Design Assistant works
- Code Assistant in action
- How to find and fix problems demo
- Q&A with Kevin Berg
Watch the Presentation
Bio
Kevin Berg is the VP of Solution Engineering at Evinced, where he is dedicated to helping leading enterprises improve and accelerate their digital accessibility efforts. Kevin trained originally as a classical musician at the University of Cincinnati and the University of Texas, where he got his doctoral degree.
He spent six years at Sauce Labs, a leading functional test automation platform, where he eventually led global solutions engineering before joining Evinced in a similar role. When he is not helping Evinced customers through challenges, Kevin is home in Montana making stuff (he is a hobby machinist), playing his oboe, and enjoying family life with his wife and two kids.
