What automated scanning can and can't catch

Automated accessibility testing is genuinely useful. It's fast, consistent, and catches a large class of real problems. It's also fundamentally limited in a way no vendor's marketing likes to admit. Understanding both halves is what separates an honest assessment from a rubber stamp.

What automated testing catches well

Machines are excellent at the checks that are unambiguous and mechanical — the presence or absence of something the code either has or doesn't:

• Images missing a text alternative entirely
• Form fields with no programmatic label
• Color contrast that falls below the measured ratio
• Missing document language or page structure
• Buttons and links with no accessible name

Our assessment runs the axe-core testing engine to catch these at scale across a representative sample of pages, and validates PDFs against PDF/UA-1. This is the fast, reliable layer.

The honest limit: roughly 57%

Industry research (Deque Systems) indicates automated tools detect roughly 57% of accessibility issues by volume. That number is worth sitting with. It means that even a perfectly clean automated scan has said nothing about nearly half of what matters — because the rest requires human judgment that no algorithm can stand in for.

What only a person can judge

The criteria a machine can't settle are exactly the ones about meaning and experience:

• Whether alt text is meaningful, not just present
• Whether the keyboard focus order makes sense to a human
• Whether instructions and error messages are actually understandable
• Whether a custom interactive component behaves correctly with a screen reader
• Whether the reading order matches the visual order

A tool can tell you an image has alt text. It cannot tell you the alt text says "image123.jpg." That gap is the whole reason human review exists.

Why this matters for any badge you trust

If a program hands out an accessibility badge on the strength of an automated scan alone, it's vouching for only the 57% — and staying silent about the rest while implying the whole. That's why our assessment pairs the automated layer with structured human review, and why we publish the limit instead of burying it. A claim is only as honest as what it admits it can't see.