The typical approach to multi-agent AI systems is choreographed. A central orchestrator assigns tasks, sequences handoffs, and controls the flow. Agent A finishes, passes to Agent B, which passes to Agent C. The orchestrator manages dependencies and handles errors. It is a factory assembly line: each agent has a defined role, and the output moves in a fixed direction.
This works for simple workflows. It collapses for complex ones. And the reason it collapses is the same reason a jazz quartet does not work like an orchestra conductor standing in front of four musicians with a baton.
The limits of top-down coordination
Top-down coordination assumes that the orchestrator knows enough to assign tasks effectively. In a well-defined workflow (extract data, transform it, load it) this assumption holds. The tasks are known, the dependencies are clear, and the sequencing is predictable.
Multi-agent systems for complex reasoning do not have these properties. When you ask a system of agents to analyse a business problem, investigate a data anomaly, or generate a strategic recommendation, the path from question to answer is not known in advance. The agents need to discover the relevant information, decide what to investigate, and adjust their approach based on what they find. A central orchestrator cannot plan this because the plan depends on the output of the work it is trying to plan.
This is where the jazz analogy becomes useful. A jazz quartet coordinates without a central conductor. There is no one assigning notes to each musician in real time. Instead, the musicians coordinate through a shared structure (the chord progression, the tempo, the form of the song) and a set of conventions about how to interact.
Shared structure, not shared instructions
In jazz, the musicians agree on the structure before they start playing. They know the key, the chord changes, the tempo, and the form (AABA, blues, etc.). This shared structure constrains the space of possible music without dictating the specific notes. Each musician can improvise freely within the structure, and the result is coherent because the structure provides the coherence.
For multi-agent systems, the equivalent is a shared context and a shared set of conventions, not a detailed task plan.
The shared context includes the goal, the constraints, the available information, and the quality standards. The conventions include how agents communicate findings, how they request information from each other, how they handle disagreements, and when they defer to another agent’s judgment.
A well-designed multi-agent system does not need a central orchestrator assigning tasks. It needs agents that share enough context to coordinate implicitly, the way musicians in a quartet coordinate by listening to each other.
Trading fours: the coordination pattern that works
One of the most useful coordination patterns in jazz is “trading fours.” Two musicians alternate playing four-bar phrases, responding to what the other played. It is structured turn-taking with a constraint (four bars) and an expectation (respond to what you heard).
For multi-agent AI, this pattern maps well. Two agents alternate producing outputs, each one building on or responding to the other’s contribution. One agent generates an analysis. The other critiques it and extends it. The first agent responds to the critique. The result is better than either agent could produce alone, not because of a central plan, but because of structured responsiveness.
The key properties that make this work: each agent has a defined perspective or capability, the turn-taking has a clear structure, and each agent genuinely responds to what the other produced rather than ignoring it and producing independent output. Without these properties, trading fours becomes two agents producing unrelated output in alternation, which is worse than a single agent.
When the orchestra model is right
The jazz analogy is not universally applicable. Some multi-agent workflows are genuinely sequential and benefit from orchestration. Data pipelines, document processing workflows, and approval chains are assembly lines by nature. The tasks are well-defined, the dependencies are known, and the value of coordination is in efficiency, not discovery.
The distinction is between workflows where the path is known and workflows where the path must be discovered. Orchestrate the known paths. Let the unknown paths improvise.
The mistake most teams make is applying the orchestra model to everything. They build a central orchestrator for every multi-agent system because orchestration feels controlled and safe. But when the problem requires exploration, discovery, and adaptation, rigid orchestration produces brittle systems that break the moment the input does not match the plan.
The practical takeaway
Design your multi-agent systems the way a jazz band designs its sets. Start with a shared structure that everyone agrees on. Define the conventions for interaction. Then let the agents respond to each other and to the problem, rather than following a script.
Monitor the output the way a musician monitors the ensemble: by listening for coherence, not by checking whether each note matched the plan. If the output is coherent, the coordination is working. If it is not, the fix is usually in the shared structure, not in the individual agents.
The best multi-agent systems, like the best jazz, sound effortless. The effort is in the design of the structure that makes the improvisation possible.
