In traditional software engineering, on-call rotations are settled practice. The team that builds the system is the team that gets paged when it breaks. This creates a feedback loop: engineers who know they will be paged at 3 AM write more reliable systems. The incentive alignment is crude but effective.
AI systems do not have this convention. Most ML engineers build models, deploy them, and hand them off. When the model degrades, when the data drifts, when the predictions go sideways, who gets paged? Usually nobody. The failure is detected weeks later in a quarterly review, by which time the damage is done.
The question of whether ML engineers should carry pagers is really a question about what kind of system an AI model is.
The case for on-call
The software engineering argument for on-call is simple: ownership. If you build it, you own it. If it breaks, you fix it. This creates accountability, better design decisions, and faster incident resolution.
For AI systems, the argument is stronger than most ML engineers want to admit. Models degrade. Data pipelines break. Feature stores produce stale data. Prediction endpoints slow down under load. These are operational incidents with real business impact, and they require someone who understands the system to diagnose and fix them.
When the ML team is not on-call, two things happen. First, incidents are handled by whoever is on-call, typically an infrastructure or platform engineer who can restart a service but cannot diagnose why the model is producing bad predictions. The system comes back online with the same underlying problem. Second, the ML team loses the feedback loop. They do not feel the pain of their design decisions, so they do not improve them. The model that was deployed without monitoring stays unmonitored because nobody who understands it is accountable for its health.
The strongest version of the argument: if an AI system is important enough to be in production, it is important enough to have someone accountable for its availability and correctness. That someone should be the team that understands the system.
The case against
The counterargument is real and worth taking seriously. AI system failures are qualitatively different from software failures, and the on-call model does not map cleanly.
When a web service goes down, the symptoms are clear: error rates spike, latency increases, users cannot access the service. The incident response is well-defined: identify the failing component, restore service, investigate root cause.
When an AI model degrades, the symptoms are ambiguous. Predictions are still being served. The endpoint is still responding. But the predictions are slightly worse, in a way that may not be detectable without domain expertise. Is a 2% drop in precision a model issue, a data issue, or normal variance? The on-call engineer cannot answer this at 3 AM.
Furthermore, many AI system failures are not fixable with a restart. A model that has degraded due to data drift needs retraining. A feature pipeline that produces incorrect values needs a code fix. These are not incidents in the traditional sense. They are development tasks that happen to have been triggered by monitoring.
Carrying a pager for problems you cannot fix at 2 AM is demoralising and counterproductive.
The middle ground: tiered response
The answer is not a binary yes or no. It is a tiered response model that distinguishes between operational incidents and model health issues.
Operational incidents (service outages, pipeline failures, infrastructure problems) should follow the standard on-call model. The team that builds the system carries the pager. If the prediction endpoint goes down, someone should be paged, and that someone should be on the ML team or the platform team that supports it.
Model health issues (drift, degraded performance, data quality problems) should follow a different model. These should trigger automated alerts during business hours with a defined response SLA. The ML team investigates during the next business day. The response is diagnosis and a plan, not an immediate fix, because the fix typically requires retraining or pipeline changes that cannot be done at 3 AM.
The distinction matters because it aligns the response with the problem. Operational incidents need immediate response. Model health issues need thoughtful response. Treating them the same, either paging someone for both or ignoring both, produces bad outcomes.
The real question
The pager question is a proxy for a deeper question: does your organisation treat AI systems as first-class production systems or as research artifacts that happen to be deployed?
If the answer is first-class production systems, they need the same operational rigour as any production system: monitoring, alerting, defined response procedures, and accountable owners. The specific on-call model can be adapted for AI’s unique failure modes, but the principle of ownership and accountability is non-negotiable.
If the answer is research artifacts, stop calling them production systems. Either elevate them to production-grade with proper operational support, or be honest that they are experiments and manage them accordingly.
The worst outcome is the middle ground: AI systems that are treated as production when leadership wants a demo and as experiments when something breaks. That is not a technical problem. It is an organisational commitment problem, and no amount of tooling will fix it.
