AI incidents: how failures are recorded, classified and learned from
AI incident databases collect real-world failures to prevent repeated mistakes. The AI Incident Database indexes harms and near harms. The AI Vulnerability Database maps failure modes to taxonomies.
Why record AI incidents
AI systems deployed in the real world sometimes cause harms, from biased outcomes to safety failures. Without a shared record of these events, organizations repeat the same mistakes. The AI Incident Database addresses this gap by indexing the collective history of harms or near harms realized by AI systems, as described on its homepage. AI Incident Database
The database also draws inspiration from aviation and computer security, where incident collections inform safety improvements. Its stated goal is to learn from experience so that bad outcomes can be prevented or mitigated. That aligns with the broader need for a collective memory of intelligent system failures.
The AI Incident Database
The AI Incident Database is a cooperative effort that supports research and development. It offers faceted and full-text search across more than 1,000 incident reports, according to the pre-print paper "Preventing Repeated Real World AI Failures by Cataloging Incidents" on arXiv. arXiv paper on AI Incident Database
Each entry typically links to news reports and describes the context, the harm, and the system involved. The database is designed to enable incident avoidance and mitigation through analysis of past failures.
The AI Vulnerability Database
The AI Vulnerability Database (AVID) is an open-source knowledge base focused on failure modes in general-purpose AI systems, including generative models and developer tools. Its stated purpose is to keep a detailed record of vulnerabilities and reports, each with evidence, metadata, and reproducible evaluation details, as described on the AI Vulnerability Database. the AI Vulnerability Database
AVID pairs this database with a taxonomy library, where the AVID taxonomy is one of several reference schemes. This supports builders who need to evaluate systems across model, tool, and application layers.
From records to risk management
Governments and international bodies also track AI incidents. The OECD.AI catalogue includes an AI Incidents and Hazards Monitor (AIM) for gaining insights on global AI incidents and hazards, along with tools for trustworthy AI development. OECD.AI catalogue
To ensure that AI developers are accountable, verifiable claims about safety, fairness, and privacy are important. The report "Toward Trustworthy AI Development" on arXiv suggests mechanisms for supporting such claims, including institutions, software, and hardware approaches. arXiv paper on trustworthy AI
| Feature | AI Incident Database | AI Vulnerability Database |
|---|---|---|
| Purpose | Index harms and near harms | Catalog failure modes |
| Scope | Real-world incidents across sectors | General-purpose AI systems and tools |
| Classification | Faceted search and taxonomies | Taxonomy library with multiple schemes |
| Evidence | Incident reports with links to news | Structured reproducible failure evidence |
| Goal | Prevent or mitigate bad outcomes | Support builders evaluating systems |
Frequently asked questions
What counts as an AI incident?
An AI incident is a harm or near harm realized in the real world by the deployment of an AI system, as indexed by the AI Incident Database.
How are incidents reported to the AI Incident Database?
The database accepts submissions from the public; users can sign up and submit incident reports through its website.
What is the AVID taxonomy?
The AVID taxonomy is a reference scheme for classifying failure modes in general-purpose AI systems, one of several in the AVID taxonomy library.
How does OECD track AI incidents?
OECD.AI provides an AI Incidents and Hazards Monitor (AIM) and a catalogue of tools for trustworthy AI.
Why is learning from incidents important?
Recording failures helps organizations avoid repeating the same mistakes and supports accountability and risk management.