DM Explains · No. 046 · Architecture
Multi-model federation: one phone number, many specialists
Running dozens of AI models at once doesn't have to mean dozens of integrations. Federation routes each request to the right model while keeping one stable interface and one accountable actor.
· 13 min read
Read me first
Multi-model federation genuinely does let an organisation run dozens of different AI models without multiplying integrations, but only when one gateway owns routing and exactly one model ever holds the authority to act.
A support-and-ops assistant fed by eight separately served Cells (f1) → 1 each Cell qualified under a versioned Profile (f2) → 2 gateway routes the ticket to its best-fit Cell (f3) → 3 read-only Analyst Cells contribute bounded evidence (f5) → 4 the sole Executor alone answers and calls tools (f6) → 5 on 20 Terminal-Bench 2.1 tasks six Cells are promoted and one rolled back locally, service stays up (f9) → 6 all eight Cells appear as one OpenAI-compatible model (f1) → 7 endpoints are locked down as the surface grows (f11)
A team has quietly accumulated a frontier model for hard reasoning, an open-weight model for sensitive data, and a small specialised model for high-volume classification — each wired directly into different apps by different engineers. Every new model release means rewiring those integrations by hand. Nobody can say which model actually took a risky action last Tuesday. One misconfigured model can become a security hole for the entire stack.
Everyday picture. The house has several specialists on call — Papa for finances, Amma for school matters, Meera for homework logistics — but there is one phone number the children dial, which connects them to whoever actually knows the answer. When Arjun asks a question at dinner, Meera and Papa may whisper hints, but only Amma decides what gets said aloud and acted on, like calling the school.
Where the picture breaks. Family members volunteer opinions freely and authority shifts by convention. In a federation, most models are strictly barred from acting at all, and only one has authority, enforced by the system rather than by social agreement.
Why it exists now
No single model wins on performance, cost, control and compliance at once, which is why enterprises have stopped betting on one model and started assembling portfolios: a frontier model for general reasoning, an open-weight model for control and economics, and smaller specialised models for edge or high-volume work, all meant to sit behind one model-independent gateway (nasscom, 2026). Nasscom's analysis is blunt about it: there is no single "best" model, because performance, control, customisation, security, complexity and cost trade off against each other depending on the task.
What makes federation workable now rather than aspirational is that engineering practice has caught up with formal structure. A 2026 paper describing the Fusion-MoA architecture shows a fleet of independently served models, called Cells, exposed as a single OpenAI-compatible model, with a versioned configuration — a Profile — governing which Cells belong and what they're allowed to contribute (arXiv, 2026). That's the difference between "we call several models from various bits of code" and "we run a federation": a registry, a routing layer, and rules about who gets to act, instead of a pile of point-to-point wires.
How it works
Registration
No model joins the fleet unannounced
What it is. Each independently served model, called a Cell, is added to a versioned Profile before it can be called by anything.
Why it exists. Without a controlled registry, adding or removing a model means redesigning every downstream application that touches it.
Where it breaks. Skip registration and there is no record of who authorised a model to participate — which defeats the entire audit purpose of building a federation in the first place.
How that's handled. Membership and evidence rights live in the Profile, not in application code, so the registry is the one place to check who's in the fleet and what they can do.
In technical terms
In technical terms: Fusion-MoA exposes independently served heterogeneous models as a single OpenAI-compatible model, with a versioned Profile controlling membership and evidence admission for each Cell.
Routing
Each request meets its best-fit model
What it is. Requests pass through a model-independent gateway that picks among frontier, open-weight and small specialised models.
Why it exists. Enterprises hold a multi-model portfolio precisely because no single model wins on performance, cost and control together (nasscom, 2026).
Where it breaks. Routing logic becomes a single point of failure or bias if it is poorly tuned or opaque — a bad router is as damaging as a bad model.
How that's handled. Routing decisions are made against task requirements, not habit: cost-sensitive, high-volume work goes to small specialised models; control-sensitive work goes to open-weight models; hard reasoning goes to frontier models.
In technical terms
In technical terms: enterprises are expected to adopt a multi-model portfolio combining frontier, open-weight and specialised models, all routed via a model-independent gateway rather than hardcoded calls.
Bounded evidence
Most models may only whisper, not decide
What it is. Read-only "Analyst" models feed bounded pieces of evidence into a task.
Why it exists. It lets weaker or specialised models contribute usefully without the power to make the final call, containing the damage any single model's mistake can do.
Where it breaks. If evidence boundaries are not enforced in code, a buggy Analyst's output can quietly become treated as authoritative by whatever reads it next.
How that's handled. The boundary is architectural, not advisory: Analysts are read-only by design, so there's no code path from "Analyst said so" to "action taken".
In technical terms
In technical terms: in Fusion-MoA, read-only Analyst models contribute bounded evidence to a task without authority over the final answer or tool calls.
Sole authority
One Executor, always
What it is. A designated Executor retains exclusive authority over final answers and tool invocations, however many models contributed evidence.
Why it exists. Centralising authority keeps the system auditable: every action traces back to one accountable component instead of an ambiguous committee of models.
Where it breaks. If more than one model can act, accountability for errors dissolves into an untraceable committee — exactly what federation exists to prevent.
How that's handled. On a 20-task evaluation using Terminal-Bench 2.1, all tool actions remained attributable to the single Executor, with zero actions taken by Analyst models and zero bypass effects (arXiv, 2026).
In technical terms
In technical terms: a sole Executor Cell retains all final-answer and tool authority within the federation; Analyst Cells have none.
Eight Cells serve a support-and-ops assistant → 1 a ninth candidate is registered → 2 on HMMT P1-P10 the collective solves 8 of 10 vs 6 of 10 for the best single Cell → 3 a new profile version admits it as evidence-only → 4 promoted into the fleet → 5 live traffic routes through the one gateway, most Cells as Analysts, one as Executor → 6 on 20 Terminal-Bench 2.1 tasks six Cells are promoted and one incompatible candidate is locally rolled back, service never interrupted → API unchanged throughout
Promotion & rollback
Models come and go, the API doesn't
What it is. Candidate Cells are qualified against benchmarks before promotion into the live fleet, or rolled back locally if they underperform.
Why it exists. This lets the federation evolve as better models appear, at the pace releases actually happen, while keeping the client-facing contract untouched.
Where it breaks. Qualification benchmarks that don't resemble real production tasks can let an unsuitable model get promoted anyway — the gate is only as good as the test.
How that's handled. In the same evaluation, six Cells were promoted into the live fleet while one incompatible candidate was rolled back locally, with the service remaining available throughout (arXiv, 2026).
In technical terms
In technical terms: Fusion-MoA Cells can be qualified, promoted, or rolled back without changing the public API that applications call.
Unified interface
One door, many rooms behind it
What it is. The federation exposes one consistent API — in Fusion-MoA's case, OpenAI-compatible — no matter how many models sit behind it.
Why it exists. Application developers need a stable contract; hiding internal diversity avoids forcing every client to know which model actually answered.
Where it breaks. Hiding diversity too well can mask real differences in model behaviour, such as varying reliability, that a caller might actually need to know about.
How that's handled. The interface stays stable while internal composition changes — this is distinct from model merging, which fuses parameters into one universal model rather than keeping them separately served (ACM Computing Surveys).
In technical terms
In technical terms: model merging combines the parameters of multiple expert models into a single set of weights without needing the original training data; federation instead keeps models separate and unifies only the calling interface.
Surface management
More doors, more doors to lock
What it is. Each added model and modality widens what must be secured and governed.
Why it exists. Federation concentrates this risk management in one layer rather than leaving every team to secure its own model integration independently.
Where it breaks. Centralising risk management in one layer only helps if that layer is itself rigorously secured; otherwise it becomes the single weak point for the entire system.
How that's handled. Industry analysis warns that adding multiple models and modalities multiplies entry points, and that a single misconfiguration or vulnerability in one integration can compromise the whole federated system if left ungoverned (F5).
In technical terms
In technical terms: a multimodal model's architecture includes numerous entry points vulnerable to exploitation, and interoperability issues mean one misconfiguration can compromise the entire system.
Compared with what came before
| Point-to-point integration | Multi-model federation | |
|---|---|---|
| Adding a new model | Rewire every app that needs it | Register once in the Profile |
| Who can act | Whoever the app developer wired in | A single designated Executor |
| Swapping a model | Edit application code | Promote or roll back behind the same API |
| Security surface | Each integration secured separately | Centralised, but a single weak point if ungoverned |
| Matches this | Model merging (one fused set of weights) | Independently served models, routed at runtime |
The judgement is simple: federation trades scattered, ad-hoc wiring for one governed layer, but that layer now carries the weight of every security and routing decision the fleet depends on — it has to be built, and secured, as seriously as the models it fronts.
Where it breaks
Not a substitute for securing the gateway itself
Federation centralises routing and authority, but it does not automatically secure them. A misconfiguration or vulnerability in one model's integration can compromise the security of the entire federated system if the gateway layer itself is not rigorously governed (F5).
Federation is also easy to confuse with three adjacent ideas, and the confusion causes real design mistakes. Model merging fuses the parameters of several trained models into one universal set of weights, without needing the original training data; federation keeps models separately served and routes between them at runtime, which is a different trade-off entirely (ACM Computing Surveys). A Mixture-of-Experts architecture is a single model's internal router choosing among expert sub-networks inside one trained network; federation operates across independently trained, often differently vendored, external models. And multi-model serving — hosting several distinct models on shared compute — is about infrastructure cost, not about who gets to answer or act; federation can sit on top of such infrastructure, but solves a governance problem, not a hosting one.
None of this fits a small team running one general-purpose model for everything: the overhead of a registry, routing layer and promotion pipeline costs more than it saves there. Nor does it fit a latency-critical edge deployment where the routing hop between separately served models is unacceptable and a single small on-device model will do.
Numbers to carry
| Figure | Detail |
|---|---|
| 8 of 10 vs 6 of 10 | Fusion-MoA's eight-Cell, three-lineage deployment solved 8 of 10 problems on a fixed HMMT P1–P10 maths slice, versus 6 of 10 for the strongest individual model (arXiv, 2026) |
| 20 tasks, zero bypass | Across 20 Terminal-Bench 2.1 tasks, all tool actions stayed attributable to the single Executor; zero actions were taken by Analyst models (arXiv, 2026) |
| 6 promoted, 1 rolled back | Six candidate Cells were promoted into the live fleet; one incompatible candidate was rolled back locally with the service remaining available throughout (arXiv, 2026) |
| 2026 | Year nasscom described enterprises moving toward multi-model portfolios routed through model-independent gateways |
Myths worth killing
| Commonly repeated | What is actually true |
|---|---|
| "Federation means merging models into one bigger model" | Federation keeps models separately served; merging fuses parameters into one set of weights, which is a different technique entirely |
| "More models always means more risk with no upside" | More models widen the attack surface, but a governed federation contains that risk in one layer instead of scattering it across every team's own integration |
| "Any model in the fleet can take action if it's confident enough" | Only the designated Executor holds authority to answer or call tools; Analyst models are read-only by architecture, not by convention |
| "Swapping in a new model means rewriting the app" | Promotion and rollback happen behind the same public API; qualified Cells can be promoted or reverted without client-side changes |
| "This is just load balancing across identical copies of one model" | The fleet is heterogeneous by design — frontier, open-weight and specialised models each handle different trade-offs, not interchangeable copies |
Worked example
- Engineers (illustrative) register eight separately served models — a frontier model, an open-weight model, and several small specialised classifiers — into the federation's versioned Profile.
- The gateway receives a support ticket and routes it to a small specialised model for triage, since this is high-volume, low-stakes classification work.
- Three Analyst models contribute bounded evidence on a harder escalation — one checks policy text, one checks account history, one checks prior tickets — none of them able to act.
- The Executor model alone reads that evidence, drafts the reply, and is the only component permitted to call the ticketing tool to send it and close the case.
- A new candidate model is qualified against the same benchmark suite the current fleet was tested against; on Terminal-Bench 2.1 style evaluation, six of eight candidates clear the bar and get promoted, one fails and is rolled back locally, with the live service never going down.
- The application team notices nothing — the same API endpoint answered the ticket before and after the promotion.
Should you use it?
Use federation when the fleet already exists in practice — several models scattered across teams — and the organisation needs one place to route, audit and swap them without rewriting every app each time a better or cheaper model turns up. Skip it when a single general-purpose model already covers every task well enough, or when the workload is latency-critical and a lone small on-device model is the only thing that fits the budget of milliseconds available. The cheapest first experiment is not building a gateway from scratch: it's drawing up the Profile on paper — listing which models exist today, which one is allowed to act, and which ones are only allowed to whisper evidence — before writing a line of routing code.
Say it in the room
"Use federation when models are already scattered across teams and need one accountable door; skip it when one model already does the job and a gateway would only add a layer to secure."
Lines to carry
- "More models, one phone number."
- "Evidence is cheap to give; authority is expensive to hold."
- "Swap the specialist, keep the number the same."
- "No model joins the fleet unannounced."
- "Centralising risk only helps if the centre itself is locked down."
FAQ
Is multi-model federation the same as a mixture-of-experts model?
No. Mixture-of-Experts is an internal architecture inside one trained model, where a router picks among expert sub-networks for each input. Federation sits above several independently trained, independently served models — often from different vendors entirely — and routes between them at the request level, not inside a single network.
Does federation make the system more secure, or less?
Both, depending on how it's built. It centralises risk management in one layer instead of leaving every team to secure its own integration separately, which is an improvement — but only if that central layer is itself rigorously secured. A misconfiguration there can compromise the entire federated system, not just one integration.
Can more than one model ever hold authority to act at once?
In the architecture described by Fusion-MoA, no — a single Executor holds that authority, and Analyst models are read-only by design. That's a deliberate choice to keep every action traceable to one accountable component, not an incidental limitation.
How is this different from just hosting several models on the same server to save money?
That's multi-model serving, an infrastructure concern about sharing compute across distinct models. Federation is a governance and routing concern — which model answers which request, and who holds authority to act — and it can run on top of multi-model serving infrastructure without being the same thing.
What happens when a new, better model comes out?
It gets qualified against the fleet's existing benchmarks and, if it clears the bar, promoted into the live fleet; if not, it's rolled back locally. In one evaluation, six of several candidates were promoted and one was rolled back, with the service staying available throughout, and the client-facing API never changed.
Glossary
- Multi-model federation — an architecture where several independently served AI models are coordinated and routed as if they were one system, without merging their parameters.
- Model merging (fusion) — combining the parameters of several trained models into a single set of weights to create one universal model.
- Mixture-of-Experts (MoE) — a single model's internal architecture where a router selects among specialised expert sub-networks for each input.
- Gateway — a layer that applications call instead of calling models directly, which decides which underlying model handles each request.
- Open-weight model — a model whose trained parameters are published so it can be run and modified independently of the original provider.
- Frontier model — one of the most capable, usually largest and most general-purpose AI models currently available.
- Small language model (SLM) — a model with far fewer parameters than large frontier models, requiring much less compute.
- Cell (in Fusion-MoA) — an independently served model participating in the federated fleet.
- Analyst (in Fusion-MoA) — a read-only model role that contributes bounded evidence but cannot take final actions.
- Executor (in Fusion-MoA) — the single model role with authority to produce the final answer and invoke tools.
- Profile — a versioned configuration that controls which models are members of a federation and what evidence they can contribute.
- Multi-model serving (MMS) — infrastructure that hosts multiple distinct models on shared server resources rather than one model per dedicated environment.
- Rollback — reverting a federation to a previous state by removing or demoting a model that failed qualification.
Sources
- The Fleet Is the Model: Engineering Collective Intelligence with Fusion-MoA Pioneer R1 — arXiv, 2026
- A Different Kind of Multiverse: Multi-Model and Multimodal Integration — F5
- AI Model Strategy For Enterprises – Choosing the right model and making it work — nasscom, 2026
- Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications, and Opportunities — ACM Computing Surveys