How Conforma tests a website
Every number this product publishes comes from the process below. It is written down so you can check it, quote it, and disagree with it.
What happens during a scan
-
Load the page in a real browser
A headless Chrome instance opens the address and waits for the page to settle, so client-rendered markup, dialogs and hydrated components are all present. Testing the HTML a server returns would miss most of a modern site.
-
Find the rest of the site
The crawler reads the sitemap where there is one and follows same-origin links where there is not. It stays on the host it landed on after redirects, so an apex-to-www redirect does not silently reduce the crawl to a single page.
-
Run axe-core against every page
Deque's axe-core is the same engine behind most browser accessibility extensions. Rules are selected by the conformance level set for the site: A, AA (the default) or AAA.
-
Record every failing element
Each failure is stored with its rule, its WCAG success criterion, its severity, the CSS selector of the element and the markup that failed, so a developer can go straight to it.
-
Score, and say what the score covers
The page score is computed from the distinct failing rules. If the crawl stopped at the plan cap or ran out of time, the result says how many pages were discovered versus tested, and the score is described as covering the pages actually tested.
Free scans cover up to 25 pages of a site; the deepest paid plan crawls up to 500. Pages can be excluded by path, and the conformance level is per site.
How the score is calculated
A page starts at 100 and loses points for each distinct failing rule, weighted by severity. Ten images missing alt text cost the same as one, because they are one fix. A site's score is the mean of its page scores. The arithmetic is fixed, so the same markup always produces the same number and the number only moves when the markup does.
Points lost per failing rule
- Critical
- −15
- Serious
- −10
- Moderate
- −5
- Minor
- −2
Grades
- A
- 90-100
- B
- 75-89
- C
- 60-74
- D
- 40-59
- F
- 0-39
Only rules that map to a WCAG success criterion count towards the score. axe-core also ships best-practice rules with no criterion behind them; Conforma runs and reports those, because they are useful, but they are advice rather than a conformance verdict.
What this cannot tell you
Automated testing reliably reaches about a third of the WCAG success criteria. A scan can demonstrate that a page fails; no scan can demonstrate that a page passes. These are the things it does not judge:
- Whether alt text describes the right thing, or merely exists
- Whether a keyboard user can complete a purchase, a booking or a sign-up from end to end
- Whether the reading order and focus order make sense to somebody who cannot see the layout
- Whether an error message explains what to do about the error
- Whether a video has captions that match what is said, or an audio description at all
- Whether a component that is technically labelled is actually understandable
Full conformance with EN 301 549, which is what the European Accessibility Act expects, needs a manual review by somebody who tests with a keyboard and a screen reader. Conforma's job is to clear everything a machine can decide, and to leave a dated record that you did.
How the claims on this site are sourced
Legal and enforcement claims name their source on the page that makes them. Where sources conflict or a case is under appeal, both sides are stated. Where nothing has happened, that is stated too: no EU regulator has issued a fine under the EAA to date, and this site says so rather than implying otherwise.
Country pages cite the transposing statute by number and name the enforcing body. The Carrefour write-up lists the decision, the reporting and the case that went the other way. Nothing here is legal advice.
Reports published about other people's sites
A scan of a public website produces a public report at /accessibility-report/{host}. Those pages always show the newest scan for that domain, they state what the scan covered, and every one of them carries a removal link that works with no account and no wait. If a report about your site is wrong or unwanted, take it down from the page itself.
Questions about the method
How much of WCAG can an automated test actually check?
Roughly a third of the success criteria, and only the machine-checkable part of those. A scanner can prove a page fails; it can never prove a page passes. Contrast ratios, missing text alternatives, unlabelled controls, language attributes and document structure are decidable by a machine. Whether the alt text describes the right thing, whether a keyboard user can complete a purchase, and whether an error message makes sense are not.
Why does the score not drop for every single failing element?
Because ten images missing alt text are one fix, not ten. The score charges for each distinct failing rule, weighted by severity, so it tracks the work in front of you rather than the size of your page. A page with one broken pattern repeated a thousand times and a page with it once score the same, which is the honest reading of the situation.
Do best-practice findings count against the score?
No. axe-core ships rules that are good advice but carry no WCAG success criterion behind them. Conforma runs them and shows them, because they are useful, but it does not count them as conformance failures. A site whose only finding is a missing main landmark is not failing WCAG, and a score that said otherwise would be inventing a compliance problem.
What happens when a crawl cannot finish?
The scan says so. A crawl that hit the page cap or ran out of time reports how many pages it discovered and how many it tested, and the score is described as covering those pages rather than the whole site. Reporting a partial crawl as if it were a complete one is the single easiest way for a tool like this to lie.
Can I have a published report about my site removed?
Yes, from a link on the report itself, with no account and no wait. Reports are generated only for domains somebody actually scanned, they always show the newest scan, and they carry the removal link at the foot of the page.
Run the method against your own site
Free, no sign-up, and the result is yours to keep or share.
Related: the fix guide for every rule we report, what the EU Accessibility Act requires, and Conforma's own accessibility statement.