This tool automatically checks your JavaScript web application for digital accessibility.
The application is scanned for all relevant subpages using an autonomous crawler.
These pages are then checked for compliance with WCAG 2.2 based on the axe-core engine and additional rules.
The results are stored as full-accessibility-audit.json and full-accessibility-audit.md in the full-accessibility-audit artifact. Violations of level A and AA fail the workflow. Findings beyond that, like level AAA (e.g. generic link texts) or best practices (e.g. skipped heading levels), are listed separately as hints and do not fail the workflow.
If none of the start pages can be crawled, e.g. because the app is not reachable or returns an error, the workflow fails. If there is nothing to audit, e.g. because all pages are excluded, the workflow passes and the report states that no pages were audited.
To use the finalized test environment, you must either create a new .yml file in the .github/workflows directory or edit an existing .yml file.
For the specified jobs, please insert the following code:
jobs:
full-accessibility-audit:
uses: StackOverflowIsBetterThanAnyAI/full-accessibility-audit/.github/workflows/full-accessibility-audit.yml@master
with:
app_port: "5173" # specify the port of your app
start_command: "npx vite --port 5173 --host" # specify the start command for your app with port and host flags being set
node_version: "22" # optional, default: "24"
skip_crawler: true # optional, default: false
base_path: "/path" # optional, default: ""
config_file: "config.json" # optional, default: "sitemap.json"
image_tag: "1.2.0" # optional, default: "latest", also "sha-<commit>" works
secrets:
audit_env: ${{ secrets.AUDIT_ENV }} # optional, values for the placeholders of the state
This ensures that the code is checked for digital barriers every time the file is run in this GitHub Actions pipeline.
The audit runs inside a Docker container and reaches your app via the IP address of the runner. Therefore your app has to listen on all network interfaces instead of localhost only, e.g. with --host for Vite or -H 0.0.0.0 for Next.js.
The optional config file cypress/fixtures/sitemap.json (or the name given in config_file) controls which pages are audited:
{
"config": {
"only": [],
"included": ["/landing-page"],
"excluded": ["/admin/*", "/logout"],
"queryParams": ["page"],
"maxPages": 500
}
}
| Option | Description |
|---|---|
only |
Audits exactly these paths without crawling. included is ignored. |
included |
Additional start pages for the crawler, e.g. pages that are not linked anywhere. They are audited and their links are crawled. |
excluded |
Patterns of pages that are neither visited nor audited, e.g. logout links. Always takes priority over only and included. |
queryParams |
Query parameters that lead to separate pages. By default, urls that only differ in their query parameters count as one page. |
maxPages |
Maximum amount of pages the crawler visits, default: 500. |
All paths are relative to base_path. Patterns in excluded support * as a wildcard for any characters including /:
/admin/*excludes/admin/usersand/admin/users/edit, but not/admin./admin*excludes both./logoutalso excludes/logout?redirect=1and/logout#/confirm. Query parameters and hash routes only matter if the pattern contains them, e.g./shop?print=*.- The comparison is case-insensitive and ignores trailing slashes.
Hash routes like #/about or #!/about are treated as separate pages, anchors like #section are not. Tracking parameters like utm_source or fbclid are ignored.
The crawler stores the found pages as urls in the same file. With skip_crawler: true, these urls are audited directly.
Pages that are only available with a certain state, e.g. after a login or with an accepted cookie banner, can be reached with an injected state:
{
"config": { ... },
"state": {
"localStorage": { "token": "{{env.AUDIT_TOKEN}}", "settings": { "theme": "dark" } },
"sessionStorage": { "step": "2" },
"cookies": [
{ "name": "consent", "value": "accepted" },
{ "name": "session", "value": "{{env.AUDIT_SESSION}}", "httpOnly": true }
],
"indexedDB": [
{
"database": "app",
"version": 1,
"stores": [{ "name": "users", "keyPath": "id", "records": [{ "id": 1, "name": "Ada" }] }]
}
]
}
}
| Option | Description |
|---|---|
localStorage |
Entries of the local storage. Objects are stored as JSON. |
sessionStorage |
Entries of the session storage. Objects are stored as JSON. |
cookies |
Cookies with name and value, optional domain (default: host of the app), path, secure, httpOnly, expiry and sameSite. |
indexedDB |
Databases with version and stores. Each store has a name, optional keyPath, autoIncrement, indexes and records. Without keyPath, records are { "key": ..., "value": ... }. |
The crawler always uses the state, so it also finds the pages that require it. Every page is audited twice, once without and once with the state. Pages that the server only delivers with the state (e.g. 401 without the session cookie) are only audited with the state. Logout links should be added to excluded, otherwise the crawler might delete the session.
If a page redirects with the state to the same location as without it, e.g. to the login because a token has expired, the state was probably not accepted. The crawler would lose such a page and the audit would only check its version without the state. These pages are listed as warnings in the sitemap, in full-accessibility-audit.json and at the top of full-accessibility-audit.md. Warnings do not fail the workflow. A page that stays at its url and only shows different content (e.g. a login form on /) cannot be detected this way.
Before every page load of the audit, cookies, local storage, session storage, IndexedDB, cache storage and service workers are cleared, so data the app stored on the previous page does not affect the next one.
The IndexedDB databases are recreated before every page load, so the app does not run its own upgrade. Therefore version has to match the version of the app and stores has to contain all object stores and indexes the app expects.
Secrets must not be stored in the config file. Use placeholders like {{env.AUDIT_TOKEN}} instead and pass the values as the secret audit_env with one KEY=VALUE per line:
AUDIT_TOKEN=eyJhbGciOi...
AUDIT_SESSION=4f2a...
The placeholders are only replaced during the audit, the sitemap written by the crawler still contains the placeholders. The audit fails if a placeholder has no value.
The audit can be validated against the official W3C ACT rules test cases. The benchmark is a development tool and is neither part of the audit workflow nor of the Docker image.
npm run benchmark:fetch # download all test cases
node benchmark/fetch-testcases.mjs --rules=bc4a75 # or only selected ACT rules
npm run benchmark
The results (TP / FP / TN / FN, precision, recall and false positive rate per ACT rule) are written to benchmark/results/.
Every W3C test case is expected to either fail or pass (or be inapplicable). A test case counts as detected if the audit reports one of the checks mapped to its ACT rule:
| Value | Meaning |
|---|---|
| TP (true positive) | The test case is expected to fail and the audit detects it. |
| FP (false positive) | The test case is expected to pass or is inapplicable, but the audit reports a violation. |
| TN (true negative) | The test case is expected to pass or is inapplicable and the audit reports no violation. |
| FN (false negative) | The test case is expected to fail, but the audit misses it. |
| Precision | TP / (TP + FP): share of the reported violations that are real violations. |
| Recall | TP / (TP + FN): share of the real violations that are detected. |
| False positive rate | FP / (FP + TN): share of the passing or inapplicable test cases that are wrongly reported. |
| Mapped / unmapped | Test cases of ACT rules with or without a mapped check. Unmapped test cases are not part of TP / FP / TN / FN. |
| Errored | Test cases that could not be audited, e.g. because the page failed to load. |
The results of the last daily run of the regression tests:
| Test cases | Mapped | TP | FP | TN | FN | Precision | Recall | False positive rate |
|---|---|---|---|---|---|---|---|---|
| 1194 | 850 | 207 | 0 | 574 | 69 | 100.00 % | 75.00 % | 0.00 % |
344 test cases belong to ACT rules without a mapped check, 0 test cases errored.
npm run benchmark:axe runs the same test cases with axe-core only: the custom checks are skipped and the axe-core rules replaced by them are enabled again. npm run benchmark:compare-axe compares both runs per ACT rule and writes benchmark/results/act-benchmark-vs-axe.md.
| axe-core only | Full audit | |
|---|---|---|
| testcases | 1194 | 1194 |
| mappedTestcases | 798 | 850 |
| unmappedTestcases | 396 | 344 |
| erroredTestcases | 0 | 0 |
| TP | 182 | 207 |
| FP | 2 | 0 |
| TN | 537 | 574 |
| FN | 77 | 69 |
| precision | 98.91 % | 100.00 % |
| recall | 70.27 % | 75.00 % |
| falsePositiveRate | 0.37 % | 0.00 % |
ACT rules with different outcomes (axe-core only → full audit):
| ACT Rule | Name | TP | FP | TN | FN | unmapped | error |
|---|---|---|---|---|---|---|---|
| 46ca7f | Element marked as decorative is not exposed | 0 → 3 | 0 → 0 | 7 → 7 | 3 → 0 | 0 → 0 | 0 → 0 |
| b4f0c3 | Meta viewport allows for zoom | 5 → 7 | 0 → 0 | 9 → 9 | 2 → 0 | 0 → 0 | 0 → 0 |
| bc659a | Meta element has no refresh delay | 4 → 4 | 1 → 0 | 9 → 10 | 0 → 0 | 0 → 0 | 0 → 0 |
| bisz58 | Meta element has no refresh delay (no exception) | 1 → 3 | 1 → 0 | 9 → 10 | 2 → 0 | 0 → 0 | 0 → 0 |
| ffd0e9 | Heading has non-empty accessible name | 0 → 8 | 0 → 0 | 7 → 7 | 8 → 0 | 0 → 0 | 0 → 0 |
| kb1m8s | ARIA global properties not used where prohibited | 0 → 5 | 0 → 0 | 0 → 4 | 0 → 0 | 9 → 0 | 0 → 0 |
| off6ek | HTML element language subtag matches language | 0 → 2 | 0 → 0 | 0 → 9 | 0 → 2 | 13 → 0 | 0 → 0 |
| qt1vmo | Image accessible name is descriptive | 0 → 0 | 0 → 0 | 0 → 13 | 0 → 3 | 16 → 0 | 0 → 0 |
| ucwvc8 | HTML page language subtag matches default language | 0 → 3 | 0 → 0 | 0 → 9 | 0 → 2 | 14 → 0 | 0 → 0 |
The workflow .github/workflows/regression-tests.yml runs every day:
- Crawler and audit tests (
npm run test:crawler): the crawler and the audit are tested against local test sites incrawler-tests/sites/, e.g. the custom checks, the injected state and hash routes. All tests must pass. - W3C ACT benchmark: the outcome of every test case is compared with
benchmark/baseline.json(npm run benchmark:compare). The run fails if a test case gets a worse outcome than in the baseline, e.g. a new false positive or false negative. Improvements and new W3C test cases are only reported. - W3C ACT benchmark (axe-core only):
npm run benchmark:axefor the comparison with axe-core. - README: if the W3C ACT benchmark has no regressions,
npm run benchmark:readmewrites the results of both benchmarks into this README. The README is only committed if the results have changed.
After an intended change of the results, the baseline has to be updated and committed:
npm run benchmark
npm run benchmark:update-baseline
The badge at the top shows how often the audit has run. Every run pulls the Docker image, therefore the workflow .github/workflows/download-badge.yml reads the total downloads of the image from its package page every day and stores them on the branch badges. The README itself does not change, shields.io renders the badge from downloads.json on that branch.