Four tools dominate accessibility testing, yet 94.8% of the top one million homepages still fail basic WCAG checks (WebAIM Million, 2025). axe-core, Lighthouse, WAVE, and Pa11y appear in almost every "best tools" list, but the tools aren't the problem. Picking the wrong one for your workflow is.
Each solves a different problem. axe-core is a testing engine you embed in code. Lighthouse is a quick browser audit. WAVE is a visual debugger that overlays issues on the page. Pa11y is a CLI-first monitoring system. Treating them as interchangeable leads to blind spots, and that's exactly why so many teams scan without actually improving.
This post breaks down what each tool catches, where each falls short, and which one fits your specific workflow. No rankings, no "best overall" nonsense, just an honest comparison backed by data from real-world studies.
For the full picture on accessibility testing approaches, see our accessibility testing tools comparison guide.
TL;DR: axe-core (full WCAG rule set, zero false positives) is the best engine for CI/CD pipelines. Lighthouse is convenient for quick checks but runs fewer audits. WAVE excels at visual, in-page review. Pa11y is ideal for ongoing monitoring. Combine at least two: automated tools catch roughly 57% of issues (Deque, 2024).
How Do These Four Tools Actually Work?
axe-core has been downloaded over 4 billion times and powers more than 13 million GitHub projects (Deque, 2025). It's not a standalone tool you install and run, it's a JavaScript testing engine that other tools build on top of. You embed it in your test suite, CI/CD pipeline, or browser extension, and it evaluates its full set of accessibility rules against the live DOM. If you've ever wondered what engine drives most automated accessibility testing, this is it.
Lighthouse takes a different approach by bundling accessibility audits alongside performance and SEO checks directly inside Chrome DevTools. Open DevTools, click "Lighthouse," and you get a score out of 100 in seconds. Under the hood, its accessibility audits are powered by axe-core and weighted by user impact (Chrome for Developers, 2025), but it runs only a subset of axe-core's full rule set. That convenience comes at a cost: fewer rules means gaps in detection that matter for compliance.
WAVE does something none of the others attempt. Instead of generating a report you read separately, it injects visual icons directly onto the page, marking errors, alerts, and structural elements in the exact place they occur. You can see where an issue lives without parsing JSON or switching between tabs. Non-developers love this approach for design reviews and content QA, but there's a significant tradeoff: no CLI, no API, no CI/CD integration. It's manual only.
Pa11y is the CLI-first option. Give it a URL, it launches a headless browser and spits out structured results you can pipe into any reporting system. What sets Pa11y apart is engine flexibility, it can run either axe-core or HTML_CodeSniffer under the hood, and you can switch between them depending on your needs. Pa11y Dashboard adds a web UI for tracking results over time, making it the only tool in this group that handles monitoring out of the box. It pulls roughly 294,000 weekly npm downloads (npm trends, 2025), modest next to axe-core's tens of millions, but it fills a niche the others don't touch.
What Does Each Tool Actually Catch?
Automated accessibility tools catch roughly 57% of total accessibility issues by volume, that's the ceiling, not the floor (Deque, 2021). Based on analysis of over 300,000 issues across 13,000 pages, that figure represents the best-case scenario when using a comprehensive rule set. No tool breaks past it because the remaining 43% requires human judgement, reading order, cognitive clarity, keyboard trap detection.
Within that 57%, the four tools don't perform equally. axe-core runs rules covering WCAG 2.0, 2.1, and 2.2 at A and AA levels, plus best-practice and AAA checks (axe-core GitHub, 2025). Lighthouse runs a curated subset. WAVE doesn't publicly disclose its rule count but organises findings into six categories: errors, contrast errors, alerts, features, structural elements, and ARIA. Pa11y's coverage depends on which engine you configure, axe-core or HTML_CodeSniffer.
A practical study by Craig Abbott tested both axe-core and Pa11y against 142 known accessibility issues. axe-core found 39 (27%). Pa11y found 29 (20%). But here's the key finding: combined, they found 49 unique issues (35%), because each tool caught problems the other missed (Craig Abbott, 2021).
Worth noting: These percentages look low, but they measure against all known WCAG failures including ones that require manual testing. Against the subset of automatable issues, detection rates are much higher. The WebAIM Million's top six violation categories, low contrast text (79.1%), missing alt text (55.5%), missing form labels (48.2%), empty links (45.4%), empty buttons (29.6%), and missing document language (15.8%), are caught reliably by all four tools (WebAIM, 2025).
If you're evaluating API-based alternatives to these tools, see our comparison of the best accessibility scanning APIs.
One thing separates axe-core from the pack: Deque's zero false positive commitment. Every issue axe-core flags as a "violation" is a genuine failure, no noise, no guessing (Deque, 2025). Uncertain findings go into a separate "incomplete" bucket for manual review. That distinction matters enormously in CI/CD, where a false positive blocks your deployment pipeline.
Where the Four Engines Actually Differ
Strip away the packaging and there are really only two detection engines in this comparison. Lighthouse runs axe-core under the hood, so it inherits axe's rules while surfacing fewer of them. WAVE and Pa11y use their own. That means "axe vs Lighthouse" is less a fight between engines than a question of how much of the same engine each one exposes.
The differences that survive scrutiny are about capability, not rule counts:
- Automatable at all? axe-core, Lighthouse and Pa11y run headless. WAVE does not. For anything involving CI, that single fact settles it.
- False positives. axe-core's zero-false-positive design means a flagged issue is a real one. Pa11y is noisier and will surface warnings you have to triage.
- Score vs findings. Lighthouse compresses accessibility into one number, which is convenient and misleading: a 95 can still hide a serious individual failure.
- Visual context. WAVE is the only one that shows you where in the rendered layout an issue lives, which is why non-developers can actually use it.
- Trend tracking. Pa11y Dashboard is the only free option here that stores results over time.
If what you actually want is a recommendation for your team and workflow rather than an engine-level comparison, that is the job of our accessibility testing tools comparison, which covers all six major tools including the commercial options. To wire whichever tool you pick into a pipeline, see accessibility testing in GitHub Actions.
From experience: We built a11yFlow's scanning engine on top of axe-core and Playwright specifically because the zero false positive guarantee matters in API responses. When a customer gets a violation from our API, it's real, they don't waste time investigating phantom issues.
Should You Combine Multiple Tools?
Each tool finds issues the others miss, that's the core takeaway from Craig Abbott's comparative study. axe-core and Pa11y combined found 26% more unique issues than either tool alone (Craig Abbott, 2021). There's no single tool that does everything.
Here's what makes sense based on team size and setup:
Solo developer or small team: Use Lighthouse for quick checks during development. Install the WAVE extension for visual review before shipping. That combination costs nothing and covers both automated and visual detection. Won't scale to CI/CD, but it's a strong starting point.
Team with CI/CD: Run axe-core in your pipeline with @axe-core/cli or through Playwright integration. Use WAVE during design review and QA. This is probably the best balance of automation and visual coverage for most teams.
Enterprise or compliance-driven: Run axe-core in CI/CD for automated gates. Deploy Pa11y Dashboard for ongoing monitoring and regression tracking. Add structured manual testing with Accessibility Insights for the 43% of issues automation can't reach. Consider an API-based scanner like a11yFlow for programmatic, on-demand scanning across large portfolios. And be wary of overlay solutions that promise one-line fixes, the data on why accessibility overlays fail is damning.
The real gap isn't between tools, it's between automated and manual. All four tools compete for the same 57% of detectable issues. The remaining 43%, reading order, keyboard traps, cognitive clarity, meaningful content alternatives, requires a human. The best tool strategy pairs one automated tool in CI/CD with a structured manual testing process.
For a deeper look at bridging this gap, see our developer's guide to web accessibility testing.
Frequently Asked Questions
Is axe-core really free?
Yes. axe-core is open source under the Mozilla Public License 2.0. It's been downloaded over 4 billion times (Deque, 2025) and is used by Google, Microsoft, and the US government. Deque sells commercial extensions (axe DevTools Pro, axe Monitor) but the core engine is free with no usage limits.
Does Lighthouse use axe-core under the hood?
Yes. Lighthouse's accessibility audits are powered by axe-core and weighted by user impact (Chrome for Developers, 2025), and run only a subset of its rules. This means Lighthouse will never catch more than axe-core on its own. If you already run axe-core in CI/CD, Lighthouse's accessibility score is redundant.
Can Pa11y test pages behind a login?
Yes. Pa11y supports custom actions that can fill in forms and click buttons before running tests. You can script a login flow in your Pa11y config. It also supports setting cookies and custom headers, which lets you pass authentication tokens directly.
Which tool is best for WCAG 2.2 compliance?
axe-core has the broadest WCAG 2.2 support, with rules spanning A, AA, and AAA levels. WAVE also supports WCAG 2.2. Lighthouse lags behind with partial coverage. But remember, no automated tool covers all of WCAG 2.2. Manual testing is required for full compliance.
How many accessibility issues can automated tools actually find?
Deque's research across 13,000+ pages found that automated tools identify 57.38% of total accessibility issues by volume (Deque, 2024). The remaining 43% requires manual testing, things like reading order, keyboard navigation flow, and whether alt text is actually meaningful.
The Bottom Line
No single accessibility testing tool catches everything. axe-core is the strongest engine for CI/CD: a full WCAG rule set, zero false positives, and over 4 billion downloads. Lighthouse is the easiest quick check but tests fewer rules. WAVE is unmatched for visual review. Pa11y fills the monitoring gap with its dashboard.
Pick one automated tool for your pipeline and one visual tool for design review. That covers the 57% automated tools can reach. For the other 43%, pair them with structured manual testing.
If you want the coverage of axe-core without managing the infrastructure yourself, our accessibility scanning API runs axe-core and Playwright behind a single REST endpoint. For a broader look at how all these tools fit into a complete testing strategy, start with our accessibility testing tools comparison guide.