Commercial aviation has a fatality rate of roughly 0.07 deaths per billion passenger miles. This is not an accident of technology. It is the result of a safety culture that was built, over decades, from the systematic analysis of failures. Every crash is investigated. Every investigation produces findings. Every finding produces changes to training, equipment, or procedure. The safety record is not a natural outcome of engineering competence. It is a constructed outcome of institutional learning.

AI systems do not have a comparable safety culture. When an AI system produces a harmful output — a biased hiring recommendation, a dangerous medical suggestion, a fabricated citation that a user relies on — the response is typically a post-mortem within the deploying organization, a blog post if the organization is transparent, and silence if it is not. There is no investigation board. There is no shared learning repository. There is no mandatory reporting system. Each organization learns from its own failures independently, which means the industry as a whole learns much more slowly than it could.

The incident reporting model

Aviation safety depends on mandatory incident reporting. Pilots, air traffic controllers, and maintenance crews are required to report incidents — not just crashes, but near-misses, equipment malfunctions, and procedural deviations. The reports are collected centrally, analyzed for patterns, and used to update safety standards. The reporting is protected: reporters are shielded from punitive consequences for honest reports, which removes the disincentive to report.

The AI industry has no equivalent. When a model produces a harmful output, the incident may or may not be logged internally. If it is logged, the log stays within the organization. If the incident is public — because a user tweets about it, because a journalist reports on it — the response is typically reactive and defensive rather than analytical and systematic.

A mandatory incident reporting system for AI would collect reports of harmful outputs, near-misses, and unexpected behaviors from deploying organizations. The reports would be anonymized to protect competitive information and analyzed for patterns. The patterns would inform safety standards that all deploying organizations could adopt. This is not a regulatory proposal. It is an engineering practice that the aviation industry has proven effective.

The checklist discipline

Aviation uses checklists for every critical procedure: pre-flight, takeoff, cruise, approach, landing. The checklists are not suggestions. They are mandatory procedures that must be completed in order, with confirmation from both pilots. The checklist discipline exists because human memory is unreliable under stress, and the consequences of forgetting a step in aviation are catastrophic.

AI system deployment has no equivalent checklist discipline. Models are deployed to production through processes that vary widely in rigor. Some organizations have comprehensive deployment checklists that include evaluation benchmarks, bias testing, latency verification, and rollback procedures. Many organizations have ad-hoc deployment processes that depend on the deploying engineer’s judgment about what checks are necessary.

A deployment checklist for AI systems would include: evaluation results on standardized benchmarks, bias and fairness assessment on known sensitive dimensions, latency and cost verification against production requirements, rollback procedure documentation, monitoring configuration confirmation, and stakeholder notification. The checklist would be mandatory, not advisory. Skipping a checklist item would require documented justification and sign-off.

The resistance to checklist adoption in AI is the same resistance that aviation faced decades ago: experienced practitioners believe their judgment is sufficient. Aviation’s response was to demonstrate, through accident analysis, that expert judgment fails predictably under specific conditions — time pressure, cognitive load, unfamiliar situations. AI system deployment involves all of these conditions. The checklist is not a substitute for judgment. It is a safety net for the predictable failure modes of judgment.

The Swiss cheese model

Aviation safety uses the Swiss cheese model of accident causation: multiple layers of defense, each with holes, aligned such that a hazard passes through all layers only when the holes in each layer happen to align. No single layer prevents all accidents. The system prevents accidents by making it unlikely that all layers fail simultaneously.

AI system safety should use the same model. The layers might include: data quality validation (catching bad inputs before they reach the model), model evaluation (catching poor performance before deployment), runtime monitoring (catching degradation after deployment), output validation (catching harmful outputs before they reach users), and human review (catching what all automated layers miss). Each layer is imperfect. The combination is robust.

Most AI systems I have seen have one or two layers — typically model evaluation and maybe runtime monitoring. They do not have the multi-layered defense that the Swiss cheese model requires. A single layer failure produces a direct path from hazard to harm. The organization then treats the harm as a model problem (“the model was not accurate enough”) rather than a systems problem (“we did not have sufficient defense layers”).

The crew resource management lesson

Aviation’s most important safety innovation may be crew resource management (CRM). Before CRM, cockpit hierarchy meant the captain’s decisions were rarely challenged. Junior officers who noticed problems stayed silent because questioning the captain was culturally unacceptable. CRM changed this by establishing that anyone in the cockpit — regardless of rank — has the authority and the responsibility to raise safety concerns.

AI teams need the equivalent. In many organizations, the senior engineer’s architectural decisions are not challenged by junior team members. The model developer’s evaluation methodology is not questioned by the deployment engineer. The data scientist’s feature selection is not scrutinized by the domain expert. Hierarchy suppresses the feedback that would catch problems before they reach production.

Creating CRM-equivalent practices in AI teams means establishing that any team member can raise a concern about a model’s behavior, a deployment decision, or an evaluation methodology without career consequences. This requires both the cultural norm (“raising concerns is valued”) and the structural support (review processes that explicitly solicit concerns from all team members, not just senior ones).

What aviation cannot teach AI

Aviation safety works because aviation is a mature industry with well-understood physics, decades of failure data, and a regulatory framework that enforces safety standards. AI is a young industry with poorly understood failure modes, limited failure data, and a regulatory framework that is still forming. The principles of aviation safety — incident reporting, checklists, defense in depth, crew resource management — transfer well. The specific implementation details do not, because the failure modes are different.

An airplane fails in physically predictable ways. A model fails in statistically unpredictable ways. The physics of flight are well-understood. The dynamics of model behavior in novel input distributions are not. AI safety practices must account for this fundamental uncertainty, which means they must be more conservative, more redundant, and more vigilant than aviation safety practices, not less.

The provocation: aviation killed people before it built a safety culture. The AI industry has the opportunity to build a safety culture before it kills people. Whether it will take that opportunity or wait for the catastrophic failure that forces the conversation is the open question.