Executive summary
Context
Superlinear and FPS BOSA, with accessibility experts Eleven Ways and AnySurfer, employ cutting edge AI to improve accessibility on federal websites for all citizens.
Goal of the project
To supercharge FPS BOSA’s accessibility check tool to recognise more accessibility failures, in order to provide clear information to citizens.
Solution
A flexible, AI-driven open-source mode integrated into BOSA’s accessibility checker. It can be used by any public agency.
Case study
The challenge
The FPS (Federal Public Service) Policy and Support (BOSA), established in 2017, assists the government and supports federal organizations in various fields including IT, HR, organizational management, integrity policy and public contracts. Within these missions, it manages citizen digital interaction (content management for federal government websites).
Eleven Ways is a Digital Accessibility Lab and a founding partner at DiAX (Digital Accessibility Experts). Its team of IAAP certified experts helps deliver websites, apps, and documents that are accessible from day one. AnySurfer offers advice, auditing, and training to help you build accessible websites. Their accessibility guidelines will improve services for all users, including those who have low vision, are hard of hearing, or have difficulties using a mouse or keyboard.
A mandate from the EU
In 2016, a new EU directive mandated that all government websites need to be accessible. This was the catalyst for change at BOSA: the FPS had to upgrade its accessibility check tool. The directive stipulated several things, for example:
Government websites and mobile applications must be accessible for everyone
How accessibility is defined (70+ criteria)
Every website and mobile application should have an accessibility statement
Every country in the EU had to transpose this directive in local legislation by 2018. The directive was to be applied in 2019 for new sites, 2020 for existing sites and in summer 2021 for mobile applications.
There are multiple criteria for what the directive means in practice but, to name a few, the contrast between the text and the background must be readable for everyone and content and applications must be accessible to people with various disabilities.
The challenge of accessibility
BOSA already used an accessibility checker tool on their websites. It was automated to a degree, and would create a list of detected issues. The tool was mostly rule-based. The problem was that a tool developed entirely with coding alone could not easily detect some accessibility issues - some of them ending up being missed entirely.
So, for example, if part of the criteria is that each image needs a meaningful description, BOSA’s tool could identify the existence of a description but couldn’t analyze whether or not that description was meaningful.
Similarly, it was relatively easy for the existing checker to notice that a specific page didn’t have a language label. Determining that the language label wasn’t the right one was way more difficult for this rule-based checker. It was useful to a point but could be improved on.
Machine learning to the rescue
This is where machine learning comes in. It can detect in seconds issues that a human would spot, but that computer code can’t. This saves people the time to manually go through hundreds of thousands of federal website pages and allows them to easily and quickly check accessibility issues on any given page(s).
BOSA wanted a solution that non-experts could use with one simple click, without having to learn all the intricate details about accessibility. The solution needed to flag up issues with any of the multiple criteria almost instantly.
This project would allow BOSA to make big steps towards achieving the accessibility required by the EU, in order to provide clear and accessible information to citizens.
The briefing
The overarching goal of this project was to improve the accessibility of government websites through AI. There were four main posts to the project, namely:
Perform contrast analysis. The contrast between text and its background or between an interactive component and its background should be at least a predefined minimum.
Language identification. Identify language of text-components and check whether they are labeled with the correct language. Aiming for 99.5% detection of wrongly labeled text-components with, at most, 0,1% false positives (language is correct but is reported as incorrect).
Image identification. The goal is to identify whether the alternative text of an image is describing the image correctly.
Classify decorative images (logo or banner) vs non-decorative images (images with important information on them).
BOSA had a strong preference for open source because of the flexibility it offers to the solution; a licence based model can quickly become prohibitive. With an open-source model, any public agency and personnel could use the tool effortlessly.
The approach needed to be very specific and well thought out. As the original tool was a rule-based accessibility checker, the AI didn’t need to fix existing errors. Rather, it needed to detect errors that the original checker simply couldn’t.
The solution
The project started with a pragmatic, phased approach, looking at things such as: can we reliably detect text on a page? Can we measure the contrast between that text and its background, and define what it means? What is the color of a background if it's an image?
Early on, it was obvious false positives could become an issue. These are accessibility issues wrongly detected by the checker. Striving for the “perfect AI accuracy” could thus result in a number of these false positives.
Getting the right balance
One of the biggest challenges was to find the right balance between good accuracy and as few false positives as possible. BOSA didn’t want to reach “unachievable perfection” but rather advance the state of the art and keep building and improving on it.
The team focused on input and submit buttons, radio buttons and checkboxes, working closely with Eleven Ways and AnySurfer, so as to comply with the EU’s accessibility rules. The accessibility experts’ critical role helped guide the team to find an excellent solution by prioritizing problems and knowing which rules had to be followed.
They helped create mock pages littered with accessibility violations, so the algorithms could be constantly tested, making it easier to have a baseline test set.
In total, the first post of the project was completed within six sprints spread over three months. Five to reinvent and refine, and the final sprint to glue everything together with BOSA’s previous checker.
Advancing the state of the art
The solution provides real added value to the existing tool. It can detect errors that BOSA’s checker couldn’t detect before, with the right balance and accuracy - in line with BOSA’s high standards.
While it’s not a huge transformation, the small but vital changes that have been added will make an enormous difference over time, especially with the new posts that will be added in the coming months
“We were very happy with the collaboration. Superlinear’s team was very attentive, and followed up feedback quickly and efficiently, ensuring that any issues were fixed. As our internal team is currently handling a lot of projects, we were very pleased with Superlinear’s expertise and agile way of working.”
Bart Declercq - Domain Architect at FPS BOSA
“We really enjoyed our collaboration with FPS BOSA. The project was very stimulating for our team. Accessibility is a very important aspect, especially for government websites, where all visitors need to be able to get easy access to the information they’re looking for. We were glad to help in that regard with our solution.”
Davio Larnout - CEO at Superlinear
“Machine Learning and AI are rapidly changing the face of Digital Accessibility. Already, advancements in the field of image recognition drastically improves the lives of millions with a visual impairment. As an exciting “bonus”, this collaboration showed that the very same underlying technologies allow us to build more advanced and more reliable tools for testing and monitoring web accessibility. This project offered all of us a promising glimpse into the future.”
Roel Van Gils - Digital Accessibility Expert at Eleven Ways
The future
As far as this project is concerned, all posts should be finished in June 2021. BOSA will continue to expand its toolsets to meet the demand of additional needs, ensuring that accessibility issues are detected quickly and early.
Their goal is to keep improving the accessibility of federal websites more and more with every project. The FPS will continue to build on the solution in the future, to further advance towards their ultimate goal.
This project also provided a framework on how BOSA can include Machine Learning and AI-based solutions into their tools. Multiple projects are currently underway to deliver accessible information to citizens.
