The Accessibility Alert That Cried Wolf: Tuning Scanner Severity So Teams Act

The accessibility monitor fires forty alerts a week and the team reads none of them. The fortieth alert was a keyboard trap on the payment page, and it sat unread for eleven days because the previous thirty-nine were a decorative image missing alt text.

An alert nobody reads is not monitoring

Monitoring is not a list of everything the scanner found. It is a promise that when something breaks for a real user, a named person learns about it in time to fix it. The moment the alert stream fills with low-value findings, that promise breaks, because the team learns the rational response to every alert is to ignore it. The fortieth alert in a noisy week is not read more carefully than the first. It is read less.

The pattern is familiar from every other monitoring discipline. A scanner reports hundreds of findings and the team tunes nothing, so the signal drowns. The accessibility version has a specific shape: automated scans find the easy stuff, missing alt text, contrast ratios, duplicate ids, and the stream fills with findings that are real but minor, while the failures that actually block a user, a keyboard trap, a form that cannot be submitted, a modal that swallows focus, arrive in the same channel at the same severity and get the same non-response.

Severity has to match what the user loses, not how confident the scanner is. A contrast ratio of 4.2 to 1 against a 4.5 to 1 threshold is a real failure and a minor one: the text is readable, the fix is a token change, and nobody is blocked today. A checkout that a keyboard cannot finish is a different class of failure entirely, and it deserves a different class of alert. If both page the same channel, the team will treat both as noise, and the checkout will stay broken. The scanner comparison makes the same point from the other side: each tool misses something, so the alert stream has to be built around what the tools can prove, not around their raw output.

Three tiers, and the test for each one

Page someone only for a failure that blocks a task. The definition is narrow on purpose: a keyboard user cannot complete checkout, a screen reader user cannot submit a form, a focus trap holds the user hostage. These are the findings that go to a named owner, with a response time measured in hours, and they are the only ones that should ever wake anyone up. If your paging tier has more than a handful of rule types in it, it is too broad, and the team will start ignoring it the same way they ignored the digest.

The daily digest takes the failures that degrade the experience without blocking it. Contrast drift after a theme update, heading order in a dynamic view, a live region that announces the wrong thing. These matter, they get fixed, and they get fixed on a schedule the team controls. The weekly digest takes the rest: the hygiene findings, the decorative alt text, the duplicate ids that change nothing for a user. Nothing in the weekly digest is fake. All of it is real. It just does not need to interrupt anyone.

The test for the tiers is the acknowledgement rate. If fewer than half the paging alerts get a human response within a day, the paging tier is too noisy and the team has already learned to ignore it. If the daily digest has items that repeat for three weeks without a fix, the routing is wrong, not the threshold: an alert with no named owner is an alert that will be ignored, whatever its severity. Route by surface and by owner, the checkout team owns checkout alerts, and never route to a shared inbox that nobody reads.

What to tune first, on Monday morning

Start with the false positives that are not false at all, the real findings that are not worth an interruption. Contrast warnings on decorative elements. Alt text on images that are already aria-hidden. Heading order warnings inside components that render their own outline. Downgrading these is not hiding failures, it is sorting them, and the count of known issues does not change when you do it.

Then check the other direction: the failures your scanner cannot see at all. A scan that reports zero errors while a keyboard user cannot check out is not a passing grade, it is a blind spot, and the zero-error scan with a blocked checkout is the proof. No amount of severity tuning fixes a gap in coverage. The alert stream can only sort what the tools find, so the tuning and the manual keyboard test have to move together, or the quiet week is the one where the real failure ships.

Frequently asked questions

Which accessibility failures deserve a paging alert?

Start with the failure the user cannot work around: a checkout a keyboard cannot finish, a form error nobody can read, a button with no name. Those are the alerts that page someone. Contrast drift and missing alt text on a decorative image go in the weekly digest, not the pager.

Is it okay to downgrade a real failure to a warning?

Yes, and it is usually the right move. Severity tuning is not hiding failures, it is sorting them. The count of known issues does not change. What changes is which ones interrupt a person, and that is the difference between a monitoring system and a noise machine.

How do I know my severity tuning is working?

Track the acknowledgement rate: what fraction of alerts get a human response within a day. If that number is under half, the alerts are noise and the tuning is wrong. Also track repeats: the same alert firing three times without a fix means the routing is wrong, not the threshold.

Who should own an accessibility alert?

Route by owner and by surface, not by rule id. The checkout team owns checkout alerts, the design system team owns component alerts, and nobody owns a shared inbox that nobody reads. An alert with no named owner is an alert that will be ignored.