This article is based on Andrew Hedges’ talk at A11yNYC.
Andrew Hedges has been building the web professionally since 1998. This is the year that section 508 passed and one year before the publishing of version 1.0 of the World Wide Web consortium’s Web Content Accessibility Guidelines (WCAG).
Fast-forward kind of two and a half decades, Andrew is the co-founder of Assistiv Labs, after an eventful career as a developer and engineering leader at Disney, Apple, and Zapier. Working at Zapier is where he fell in love with the power of automation.
It’s also where he contributed to an accessibility guild. They were doing grassroots work to improve the accessibility of Zapier’s core product. That’s where Andrew met Weston Thayer, the other founder of Assistiv Labs. Andrew, Weston, and the team wanted to fill in what they saw as gaps in accessibility in the tooling available for making the web more accessible.
As a big believer in automation, Andrew saw small and medium-sized businesses punch above their weight. At the same time, he’s a firm believer that the gold standard for having humans validate the system. That starts with the users of the software, including and especially disabled people. But in the realm of digital accessibility, that should almost always include skilled auditors.
That said, it’s not enough to get an audit, no matter how thorough or high quality, on a yearly or even quarterly basis, and rely on that alone to validate whether your product is accessible. Andrew believes one line of JavaScript will not fix 100% of a product’s accessibility problems within 24 hours. There’s a ton of work that we can offload to the machines to enable everyone to be more efficient and make better use their time to make sure that what humans build is as inclusive as possible.
To elevate your organization’s digital accessibility efforts, consider the principles of accessibility maturity models and the benefits of continuous accessibility. These frameworks and practices ensure that your digital experiences are inclusive and consistently evolving.
Understanding Accessibility Maturity Models
Andrew referred to Devon Persing’s “The Accessibility Operations Guidebook,” a guide to making accessibility work for sustainable. The book focuses on taking a more systematic approach to accessibility. In the chapter on organizations, Devon explains that accessibility maturity models help us do the following:
- Identify an organization’s current stage of maturity.
- identify gaps between the current stage and the next stage.
- Develop a plan to move from the current stage to the next stage.
Process sometimes gets a bad rap. A poorly designed process can prevent innovation. But when it’s done well, it can increase the repeatability of behaviors that lead to better outcomes.
For example, Andrew managed university residence halls. Each year, they would hire undergraduates to serve as resident advisors. These folks came in, they were bright and well-intentioned, but they had widely varying levels of skill around the things they needed to be effective.
Process is one of the tools that they used to ensure a solid baseline of outcomes. For instance, if there’s a roommate dispute and the resident advisor for the students followed the process, they were confident they’d get a decent outcome. Even if they mostly followed processes, it was going to be better than if they just winged it. This is too unpredictable.
There are various accessibility maturity models. The W3C has a Working Group that is far along in developing what’s called a group note. It’s not a specification, exactly, but it’s an accessibility maturity model.
Accessibility maturity models tend to have a similar structure. For the most part, they define dimensions for which the current stage is assessed by collecting proof points as evidence. There are several models devoted to accessibility. They all have pluses and minuses. This focuses on the W3C’s accessibility maturity model.
W3C’s model outlines seven dimensions.
- Communications
- Knowledge and skills
- Support
- Information and communications technologies (ICT) development life cycle
- Personnel
- Procurement
- Culture
All models go into organizational culture and all these things. These models try to take a holistic approach to how you assess the maturity of the organization.
The W3C model defines the following stages of maturity:
- Inactive
- Launch
- Integrate
- Optimize.
Across the various dimensions, the organization progresses through these four stages. According to experts, you can’t skip ahead to any stage. You must go from inactive to launch to integrate to optimize in that order. Each stage sets the table for the next one.
Here’s a run through of a concrete example of how this works. Under the ICT dimension, the W3C model includes a category called user research for which one of the proof points is that user research includes disabilities. If an organization doesn’t consider disability in their user research, then they would rate this proof point as no activity.
If an organization has made it as far as making plans to do this in the future, they might rate this proof point as started. If the organization inconsistently considers a subset of disability in their research, for example, maybe screen reader users only, they might rate this one as partially implemented.
If the organization consistently considers a broad range of disabilities in their user research, they can mark this proof point as complete. You can also mark them not applicable. The point of all this is to give organizations a framework they can use to better understand whether they’re getting better or worse at accessibility.
The Role of Continuous Accessibility
Continuous accessibility refers to an approach that leverages automation to make accessibility testing a seamless, ongoing process integrated into a product’s development life cycle. Instead of relying solely on periodic manual audits, continuous accessibility incorporates automated and automated-friendly tools into development workflows, ensuring that accessibility issues are identified and addressed in near real-time.
By integrating automated tools into your development processes, you can:
- Increase confidence in code quality.
- Deliver accessible experiences at scale.
- Reduce accessibility-related risks efficiently.
Automation allows machines to handle repetitive tasks, allowing your team to focus on solving hard problems. Tools such as linters and end-to-end tests can identify potential accessibility barriers early in the development cycle, ensuring issues are addressed before deployment.
This approach aligns with modern software practices like CI/CD (Continuous Integration/Continuous Deployment), where code changes are regularly tested, built, and deployed. By embedding accessibility checks into these automated processes, teams can catch accessibility issues early and efficiently.
Automation in Action: CI/CD Processes
In the old days, developers pushed changes to a staging server, manually check for quality assurance (QA), and transfer the files to a production server. It was terrifying at times. Continuous integration, CI, refers to systems that bundle software into a known working version. And on the web, continuous deployment is what updates your servers to use the new version of the software.
Continuous Integration/Continuous Deployment (CI/CD) systems have transformed software development by automating processes previously handled manually. These systems bundle software into functional versions (CI) and deploy updates to production environments (CD).
CI/CD represents two interconnected practices:
- Continuous Integration (CI): Developers integrate code into a shared repository frequently — sometimes several times a day. Automated systems then build and test the code, ensuring it is functional and compatible with existing systems.
- Continuous Deployment (CD): Once the code passes the CI stage, it is automatically deployed to production or staging environments, making it available to users with minimal manual intervention.
This automation accelerates development timelines, reduces human error, and fosters a culture of rapid iteration and improvement. Integrating accessibility tests into CI/CD ensures that accessibility issues are caught early, minimizing their impact on users. This reduces technical debt.
Integrating accessibility into CI/CD processes transforms how your organization approaches inclusion. Instead of retrofitting accessibility after a product is built, it becomes a proactive, ongoing effort embedded in every stage of development. This approach not only improves your accessibility maturity but also ensures that your users have consistent and inclusive experiences, regardless of when or how they engage with your platform.
Automatic Vs. Automated Tools
Automatic tools rely on predefined rules to flag accessibility issues. For instance, linters immediately highlight code violations within a developer’s environment. Scanners scan websites for accessibility errors like missing ARIA attributes or improper contrast ratios.
These tools are valuable for catching inaccessibility but have limitations. While they can confirm that an image has an alt attribute, for example, they cannot assess whether the description makes sense contextually.
Automated tools, on the other hand, embed accessibility checks into workflows, often encoding human judgment into the process. End-to-end (E2E) accessibility tests, for example, validate key user flows, such as logging in or submitting forms, simulating real-world usage. Tools like focus on making accessibility a central part of product testing by treating it as a first-class concern.
While automatic tools focus on catching errors based on set rules, automated tools go a step further by addressing more nuanced issues, such as ensuring consistent user flows for people relying on assistive technologies. Together, both types of tools provide a strong foundation for continuous accessibility.
While automation is valuable, it can’t stand alone. Automated tools can detect accessibility issues but often cannot fully evaluate accessibility. For example, automated scanners may identify technical issues but cannot assess the user experience for individuals with disabilities. This gap underscores the importance of manual validation by skilled auditors and end-users.
Human auditing remains essential to validating accessibility comprehensively. A combination of automated tools and manual reviews ensures that digital products meet both technical requirements and user needs.
This approach can be compared to a stack of Swiss cheese slices — automated tests are one layer, but additional layers, like manual audits and process improvements, prevent accessibility issues from slipping through. By combining automated and manual testing, you can address accessibility comprehensively.
Why Is Continuous Accessibility Compelling?
When embedded in development workflows, continuous accessibility offers benefits. By addressing accessibility early and often, teams can build better products faster while improving user experience.
One major advantage is increased confidence in code quality. Accessibility tests in CI/CD ensure that new changes don’t introduce regressions or break existing functionality. Developers can make changes confidently, knowing their code is being validated for inclusivity.
Another benefit is efficiency through automation. Machines excel at repetitive tasks, like flagging missing ARIA attributes or testing for color contrast issues, allowing human auditors to focus on solving complex problems — such as evaluating usability for screen readers or voice control.
Continuous accessibility also combats attention decay, which refers to the likelihood of developers fixing bugs quickly if they’re discovered soon after being introduced. If a developer finds an issue the same day they made a change, the context is still fresh, and they’re more likely to resolve it immediately. This reduces the risk of accessibility issues persisting for months and negatively impacting users.
Moreover, continuous accessibility largely improves an organization’s accessibility maturity. By aligning tools and workflows with maturity model dimensions, teams can track their progress and ensure accessibility is embedded throughout the product life cycle.
Finally, continuous accessibility supports scalable, sustainable accessibility. Organizations managing complex or fast-changing products can use automated tools to deliver inclusive experiences efficiently and at scale, ensuring accessibility isn’t treated as a one-off effort.
Tying Continuous Accessibility to W3C Proof Points
Continuous accessibility streamlines development processes and improves an organization’s accessibility maturity. Using the W3C Accessibility Maturity Model as a framework, Andrew highlighted specific proof points under the Information and Communication Technology (ICT) development life cycle dimension that continuous accessibility impacts.
For example:
- Developer’s accessibility checklists: Automated tools like linters serve as real-time checklists within a developer’s environment, flagging common accessibility issues before code is even committed.
- Consistent approach to implementing accessibility features: Continuous accessibility validates the baseline functionality of accessible features across products, ensuring consistency.
- Testing process includes automated accessibility testing: Integrating accessibility checks into CI/CD workflows satisfies this proof point completely by catching issues as part of an automated build-and-deploy process.
- Accessibility as a product release gate: While few organizations currently fail a release based solely on automated test results, continuous accessibility sets the groundwork for achieving this in the future.
By addressing these proof points, continuous accessibility can elevate an organization’s accessibility maturity score:
- Starting from 0 out of 300 points, integrating automated tools can achieve a measurable 40 out of 300 points.
- Including related proof points, such as prioritization systems for accessibility issues and integrating accessibility into bug tracking, increases the score to 68 out of 300 points.
- Future advancements, such as incorporating continuous design workflows, could push the total to 108 out of 300 points — a meaningful 36% improvement.
This scoring system demonstrates the tangible impact of continuous accessibility on organizational maturity and reinforces its role in creating sustainable, scalable accessibility processes.
The Importance of Triage and Prioritization
While automated tools are powerful, their effectiveness depends on how teams handle the results. Tools like linters and scanners often generate a high volume of reported issues. Without a clear process for triage and prioritization, teams risk becoming overwhelmed by noise.
To make automated results actionable, organizations should:
- Implement a structured system for triaging accessibility issues, distinguishing high-impact problems from minor ones.
- Integrate accessibility issues into bug tracking systems with a dedicated category, ensuring they are prioritized alongside other critical defects.
- Regularly refine processes to eliminate duplicate or low-priority issues that can distract teams from meaningful fixes.
By managing automated results thoughtfully, organizations can focus their efforts where they matter most: addressing barriers that significantly impact users.
The Value of End-to-End Accessibility Testing
While linters and scanners are effective for detecting individual issues, they lack the ability to validate user flows — how people interact with and navigate through a product. This is where end-to-end (E2E) accessibility testing becomes essential.
End-to-end tests simulate real-world scenarios, such as:
- Logging into an application.
- Submitting a form.
- Completing multi-step tasks using keyboard navigation, screen readers, or voice commands.
By treating accessibility as a first-class concern, E2E testing ensures that entire workflows are accessible, not just isolated components. However, Andrew noted that traditional E2E frameworks often carry ableist assumptions, such as reliance on mouse inputs or incomplete emulation of keyboard events. Addressing this gap, tools focus on building tests that respect how assistive technology users interact with digital products.
Combining E2E testing with automated tools enables teams to validate both technical compliance and functional usability, ensuring a better experience for all users.
Shifting Accessibility Left: From Development to Design
As tools and workflows evolve, accessibility is beginning to shift earlier in the development process — into the design phase. This approach, often called “shifting left,” helps teams identify and address accessibility issues before they’re even coded.
For example, Stark’s accessibility plugin for Figma allows designers to check color contrast, typography, and other accessibility criteria during the design process. Continuous design workflows integrate accessibility tools into design systems, enabling real-time checks similar to CI/CD processes for code.
By catching issues during the design stage, teams reduce costly rework later in development and ensure that accessibility is considered from the very start. Shifting left represents the future of continuous accessibility, making inclusive design a proactive, integral part of the product life cycle.
Layered Testing: The Swiss Cheese Analogy
No single method of accessibility testing can catch every issue. As previously mentioned, Andrew described the layers of testing as a stack of Swiss cheese slices. Each layer has holes, but when stacked together, they provide comprehensive coverage.
- Automated tools (linters, scanners) identify rule-based issues, such as missing ARIA labels or insufficient color contrast.
- End-to-end tests validate entire user workflows, ensuring real-world usability for assistive technology users.
- Manual audits by skilled testers and disabled users provide the final layer, evaluating subjective elements like the intent and clarity of design.
While each layer has limitations, combining them ensures that fewer issues slip through the cracks. This layered approach reinforces the importance of balancing automation with human expertise to deliver truly accessible products.
The Balanced Approach: Tools and Human Processes
Automation alone cannot achieve accessibility. While automated tools are essential for speed and scalability, they must be complemented by human judgment to address the full range of accessibility challenges. Manual testing, user research, and process improvements provide the context and insight that machines lack.
Continuous accessibility works best when teams:
- Leverage automated tools to identify and resolve technical issues efficiently.
- Implement E2E tests and manual audits to validate user experiences.
- Integrate accessibility into design, development, and release processes.
By combining automated testing with human oversight, organizations can ensure that their digital products are not only technically compliant but also genuinely usable for people with disabilities.
Bringing Everything Together
Continuous accessibility transforms how organizations approach digital inclusion. By integrating accessibility checks into automated workflows, teams can catch issues early, improve efficiency, and build more inclusive products at scale. This approach aligns with accessibility maturity models, enabling measurable progress through tangible proof points.
However, tools alone are not enough. Teams must thoughtfully prioritize automated results, embrace end-to-end testing, and layer automation with manual validation. As accessibility shifts earlier into design, the future promises even greater opportunities to build inclusivity into every stage of the product life cycle.
With continuous accessibility, organizations can move beyond compliance and create digital experiences that work for everyone — consistently, sustainably, and at scale.
Video Highlights
- Understanding the accessibility maturity model
- The role of continuous accessibility
- Testing is like a stack of Swiss cheese slices
- Automatic versus automated
- Q&A with Andrew Hedges
Watch the Presentation
Resources
- Andrew’s slides
- The Accessibility Operations Guidebook by Devon Persing
