Google’s 2015 paper “Hidden Technical Debt in Machine Learning Systems” described a problem that has only gotten worse in the decade since. The paper’s central observation was that the model itself is a tiny fraction of an ML system’s codebase, and the surrounding infrastructure — data pipelines, feature engineering, serving layers, monitoring, validation — is where the debt accumulates. The model is the ice cream. The system is the glacier.

Most organizations treat ML technical debt the way they treat traditional technical debt: as a backlog item that will be addressed when there is time. This approach fails for ML systems because ML debt is qualitatively different from traditional software debt. It is harder to detect, harder to attribute, and harder to repay. Organizations that do not account for these differences will find their ML systems becoming increasingly expensive to maintain, increasingly fragile in production, and increasingly resistant to improvement.

Why ML debt is different

Traditional technical debt manifests as code complexity, outdated dependencies, and architectural mismatches. These are visible to engineers who work with the code. An engineer who reads a tangled function recognizes the debt. A code review that flags an outdated dependency identifies the debt. The debt is in the code, and the code is inspected regularly.

ML debt is partially invisible to code inspection. The model weights are not readable. The feature engineering logic may be correct in isolation but produce problematic interactions with the data distribution. The training-serving skew is not visible in either the training code or the serving code — it is visible only in the difference between them, which requires comparing two systems that are maintained separately.

Consider a concrete example. A fraud detection model uses a feature called “transaction velocity” — the number of transactions in the last hour. The training pipeline computes this feature from historical data with a one-hour window. The serving pipeline computes it from live data with a one-hour window. Both implementations are correct. But the training pipeline uses UTC timestamps, and the serving pipeline uses local timestamps because the serving system was built by a different team that inherited a legacy configuration. The feature values differ by the time zone offset. The model still works — the velocity signal is strong enough that the offset does not destroy its predictive power — but the accuracy is measurably worse than it should be, and the degradation is invisible because the feature values look reasonable.

This kind of debt is not detectable by code review. It requires comparing the behavior of two systems in production, which requires the instrumentation to capture that comparison, which requires someone to think about the comparison as a possibility. Most organizations do not have this instrumentation, which means the debt exists undetected and compounds silently.

The five categories

ML technical debt falls into five categories, and each one has characteristics that make it harder to address than its traditional software equivalent.

Data debt. Training data accumulates staleness, bias, quality degradation, and schema drift. Unlike code debt, data debt cannot be fixed by refactoring. It requires re-acquiring, re-cleaning, or re-labeling data, which is expensive in both time and money. The cost of data debt is paid in model accuracy degradation, and the degradation is gradual enough that it is attributed to “the model getting old” rather than to the data pipeline producing stale or biased inputs.

Feature debt. Feature engineering logic accumulates complexity as new features are added and old features become unused but are not removed. Features interact in ways that are not documented and not understood by anyone on the current team. Removing a feature risks breaking a model that depends on it, but keeping the feature adds serving latency and maintenance burden. The debt grows because adding features is easy and removing them is risky.

Infrastructure debt. The ML infrastructure — training clusters, feature stores, model registries, serving systems — accumulates configuration drift, outdated dependencies, and architectural shortcuts taken under deadline pressure. This debt is similar to traditional infrastructure debt, but it is more expensive to repay because ML infrastructure has more dependencies and fewer standardized operational patterns.

Monitoring debt. ML systems need monitoring for both system health (latency, throughput, errors) and model health (accuracy, drift, bias). Most organizations monitor system health and underinvest in model health monitoring. The debt is the gap between what you can see about your model’s behavior in production and what you need to see to detect degradation before it affects users.

Organizational debt. The team structure, ownership model, and operational practices around ML systems accumulate debt as the systems mature. A model that was built by one person and is now critical infrastructure may have no documented ownership, no runbook, and no on-call rotation. The organizational debt is the gap between the system’s operational maturity and the organization’s operational practices for that system.

Measuring the debt

The first step in managing ML debt is measuring it, and the measurement is harder than for traditional software debt because there is no equivalent of cyclomatic complexity or dependency freshness.

I use four proxy measurements.

Model accuracy over time. Track your model’s evaluation metrics on a held-out test set, refreshed monthly. If accuracy is declining, you have data debt or feature debt, and the rate of decline is the interest rate on the debt.

Incident frequency and resolution time. Track how often ML-related incidents occur and how long they take to resolve. Increasing frequency or resolution time indicates infrastructure debt or monitoring debt.

Time to retrain. Measure how long it takes to retrain and redeploy the model from scratch. If this time is increasing, your infrastructure debt is growing.

Team cognitive load. Ask the team how much of their time is spent on maintenance versus new development. If the maintenance fraction is above fifty percent, your debt is consuming more capacity than your development.

Repayment strategies

The organizations that manage ML debt well share three practices.

Budget for debt repayment explicitly. Allocate a fixed percentage of ML engineering capacity — typically twenty to thirty percent — to debt repayment. This is not capacity that could be used for new features. It is capacity that is already being consumed by the debt, either visibly through maintenance work or invisibly through degraded model performance. Making the allocation explicit makes the cost visible and the repayment intentional.

Prioritize debt by its compound rate. Not all debt compounds at the same rate. Data debt compounds faster than infrastructure debt, because data quality degrades continuously while infrastructure degrades only when changes are made. Prioritize the debt that compounds fastest, because it is the debt that will become most expensive to repay if left unaddressed.

Build debt detection into the development process. Every model change should include a check for new debt. Did the change add features without removing unused ones? Did the change introduce a dependency that will age? Did the change add complexity to the training pipeline without adding corresponding monitoring? Making debt detection part of the development workflow prevents new debt from accumulating faster than old debt is repaid.

The honest assessment

If you have an ML system in production and you do not know how much technical debt it carries, you have more debt than you think. The debt is there. It is accumulating. And it is costing you in model accuracy, incident frequency, team capacity, and time-to-market for new features. The question is not whether you have debt. The question is whether you are measuring it, managing it, and budgeting for its repayment.

The organizations that will have the most capable ML systems in three years are not the ones building the most models today. They are the ones managing the debt in the models they have already built. Accumulation is easy. Maintenance is hard. And the gap between the two is where competitive advantage in ML actually lives.