In 2023, I rewrote a data pipeline three times because the framework landscape kept shifting. First it was built on LangChain. Then the team wanted to switch to LlamaIndex because it handled retrieval better. Then a newer framework emerged that promised to unify both. Each rewrite took two to three weeks. Each rewrite produced a system that worked roughly as well as the one it replaced. The net effect of six months of framework migration was zero improvement in the pipeline’s actual output.

That experience broke something in me. I stopped evaluating new frameworks when they appeared and started evaluating them only when a specific, measurable problem required a capability that my current stack did not provide. The discipline felt constraining at first. Within three months, my team was shipping features faster than teams that were still chasing frameworks, because we were building on a stable foundation instead of perpetually rebuilding on shifting sand.

The cost of framework churn

Framework churn has costs that are real but rarely measured.

Learning curve time. Every new framework requires time to learn its abstractions, its configuration model, its error patterns, and its undocumented behaviors. The learning curve is not just the time spent reading documentation. It is the time spent debugging unexpected behaviors, discovering limitations that are not mentioned in the marketing, and developing the intuition for when the framework’s abstractions leak.

Migration risk. Moving from one framework to another always introduces bugs. Not because the new framework is worse, but because the migration requires re-implementing logic in a different abstraction model, and the re-implementation inevitably handles edge cases differently. These differences are subtle and often undetected until production.

Team fragmentation. When different team members have different levels of familiarity with different frameworks, the team’s effective capacity decreases. Code reviews take longer because reviewers are less familiar with the framework. Knowledge is less transferable because each framework has its own patterns. Onboarding new team members takes longer because there is more surface area to cover.

Architectural incoherence. When a system is built on multiple frameworks — because different parts were built at different times when different frameworks were popular — the system’s architecture reflects the framework landscape’s history rather than the system’s actual requirements. The architecture is an archaeological record of technology trends, not a coherent design.

When to adopt a new framework

I do not advocate for never adopting new tools. I advocate for adopting them when specific conditions are met, and those conditions are more demanding than “the new framework looks better.”

Condition one: you have a specific, measurable problem that your current framework cannot solve. Not a problem that the new framework solves more elegantly. A problem that your current framework cannot solve at all, or that it solves with unacceptable performance, reliability, or maintenance characteristics. If the current framework handles the problem adequately, the elegance of the new framework’s solution is irrelevant.

Condition two: the new framework has been stable for at least six months. New frameworks are volatile. APIs change. Documentation is incomplete. Community knowledge is thin. Adopting a framework in its first six months means accepting the risk of breaking changes, undocumented behaviors, and insufficient community support. Waiting six months filters out the frameworks that generate initial excitement but fail to sustain development.

Condition three: the migration cost is lower than the expected benefit, measured over twelve months. This requires estimating both the migration cost and the expected benefit in concrete terms. “The new framework is better” is not a concrete benefit. “The new framework reduces our inference latency by 40%, which enables a use case that our current framework cannot support within our latency budget” is a concrete benefit. If you cannot state the benefit in specific, measurable terms, you do not have a sufficient reason to migrate.

Building on stable foundations

The alternative to chasing frameworks is building on stable foundations and investing in the layers that do not change.

The data layer is stable. Your data storage, your data quality practices, and your data governance model will outlast any AI framework. Invest in these. A well-designed data layer can be served by any framework. A poorly designed data layer will constrain you regardless of which framework you choose.

The evaluation layer is stable. How you measure model quality, how you evaluate output correctness, and how you monitor for regression — these practices are independent of the framework. A rigorous evaluation framework is more valuable than a sophisticated model development framework, because it allows you to compare frameworks objectively rather than choosing based on marketing.

The integration layer is moderately stable. How your AI system connects to the rest of your infrastructure — APIs, queues, databases, monitoring — should be designed to be framework-agnostic. This means using standard interfaces (REST APIs, message queues, standard data formats) at the boundaries between your AI system and the rest of your infrastructure. The framework-specific code should be contained within the AI system’s boundary, not leak into the integration layer.

The heresy

The heresy is this: most AI frameworks are thin wrappers around API calls. They add abstraction layers that simplify common patterns and complicate uncommon patterns. When the common pattern matches your use case, the framework saves time. When it does not — which is often, because real-world use cases are idiosyncratic — the framework adds complexity without adding value.

The teams I see shipping the most reliable AI systems are the ones that use minimal frameworks. They call the model APIs directly. They write their own orchestration logic. They build their own evaluation harnesses. This approach requires more code upfront but produces systems that the team fully understands, can debug without fighting framework abstractions, and can modify without waiting for a framework update.

I am not arguing that frameworks are always wrong. For rapid prototyping, for teams without deep engineering capacity, and for common use cases that map well to a framework’s abstractions, frameworks are valuable. I am arguing that the default assumption — that you should use a framework because one exists — is wrong. The default should be to use the simplest tool that solves your specific problem, and to upgrade to a framework only when the simple tool’s limitations produce measurable costs.

The heuristic: if you have spent more time learning a framework than the framework has saved you, you are behind. Measure it honestly. If the answer surprises you, act on the surprise.