Yes, you should test to standards. Now what?

Chris Keroack
October 8, 2015
While testing the Equal Entry website, Voiceover displays the following message: "You are currently on a link, inside of a list. To click this link, press Control-Option-Space.
Testing with VoiceOver on Mac

Testing software, hardware, or websites to accessibility standards can often be a classic “peeling back layers of an onion” problem. You know you need to make your offerings accessible, but which industry and government standards do you need to follow? You know which guidelines you will follow, but which test tools and best practices will you use? You have a test plan, and have done some of the testing, but which issues will you fix, and how? You’ve tested to standards, but are your offerings truly accessible?

Let’s take a moment to give you some starting points to answer these questions.

Which standards?

  • In general, for U.S. software and hardware, you can use the Section 508 regulations.
  • For offerings in the EU, EN 301 549 [pdf] is the emerging standard. It borrows from and builds on Section 508 and WCAG to consolidate guidance for software, hardware, and the web.
  • For websites, the international industry standard is WCAG 2.0.
  • Section 508 and other government regulations are in the process of refreshing, to sync to WCAG, and reflect advances in technology since Section 508 was first created at the turn of the century.

Which tools and best practices?

  • Get educated on the basics of accessibility testing and accessibility policy through online training. SSB BART Group, for example, offers such online courses (including several produced here at Equal Entry) via SSB University.
  • Apply accessibility principles around the structure of content, metadata for images, media, and documents, simple UI interfaces, color, and more at design time. As with any other testing, the earlier the issues are found, fixed, or prevented altogether, the easier it is to resolve issues quickly.
  • Automated testing can provide a first pass to find simple or obvious issues, general bug trends, and help you get familiar with accessibility standards too. One example for the web is the Accessibility Management Platform from SSB BART Group.
  • Debugging tools can help you with further automated testing, and with pinpointing issues page by page and element by element. One example for the web is aXe: the Accessibility Engine by Deque. Another for color contrast issues is Colour Contrast Analyser by Paciello group.

Which issues will you fix, and how?

  • Just as with tools and best practices, get educated on the basics of accessibility development and accessibility policy through online training. As previously stated, SSB Bart Group offers online courses via SSB University.
  • Enlist accessibility experts to assist with testing, triage, and remediation of your offerings’ accessibility issues. We at Equal Entry offer such services; contact us for more information.
JAWS displays a links list
Testing with JAWS on Windows

You’ve tested to standards, but is your product truly accessible?

Nope! Apologies for burying the lede, but testing to standards alone isn’t enough.

  • For one thing, the standards themselves might be forward-looking enough that the industry experience and best practices aren’t there yet. For example, let’s say you’re attempting to produce a test plan to assess compliance with aspects of EN 301 549.
    • There are many clauses that are simply informative, e.g., (at the time of this writing) 11.2.2.7. The tester is met with “Clause 11.2.2.7 is informative only and contains no requirements requiring test.” Now the tester will need to do extra work to consider: How and when is this integrated into testing? What conversations need to happen with developers or designers?
    • Or look up 9.2.1 – the testing guidance tells the tester to refer to WCAG Success Criterion 1.1.1 – thus sending them on a wild chase to try to consolidate their test plan.
  • For another thing, your offerings may be close to technically accessible according to automated testing… yet automated testing is not comprehensive. Automated tests won’t verify full keyboard access, or catch whether the reading order of major sections of a document is correctly read by assistive technology, for example.
  • Speaking of assistive technology: even adhering closely to technical standards doesn’t guarantee a good experience with assistive technology (AT).
    • Browser / OS compatibility, compatibility issues with common AT settings and features, even fundamental usability issues like a feature with drag-drop operations that doesn’t have a keyboard equivalent… all these and more aren’t found without some expert manual testing. Be sure to leave time and budget to plan for manual testing, with common AT and browser/system/app settings, such as the testing services provided by Equal Entry.

Accessibility testing is as complex and varied as it is vital to ensure you’re making your offerings usable to all of your potential customers. But you don’t have to tackle it alone! Use the guidelines and resources mentioned here to get prepared, and to get help.

Chris Keroack is an accessibility consultant with Equal Entry.

Chris Keroack