When an AI failure becomes a disclosable incident

Operational resilience frameworks were written with outages and cyber events in mind. Supervisors are now applying them to a third category: a model that continues to run but produces materially wrong output. The distinction matters because a silent failure does not trigger the monitoring that an outage does, and can persist for weeks.

Most incident response policies do not mention models. They define incidents in terms of availability and confidentiality, with integrity handled as a data issue. A model that degrades — because input distributions shifted, because an upstream provider changed a version, because a prompt was edited — fails none of those tests cleanly while producing exactly the harm the framework exists to prevent.

The materiality test is the decision to make in advance. What magnitude of error, affecting what population, over what period, constitutes an incident requiring escalation and potentially notification? Organisations that decide this during an incident reliably decide it too narrowly, because the people deciding are the people who would have to make the notification.

Third-party dependency deserves separate attention. Where a provider changes or deprecates a model version and your output changes with it, that is your incident in the eyes of anyone affected. Standard contractual notification periods for version changes are frequently shorter than the time you need to revalidate. That mismatch should be negotiated, and rarely is.