A team routing queries across a coding specialist, a logic specialist, and a generalist model assumes each will cover the others' blind spots. A new study evaluating 67 frontier models from 21 providers shows that assumption is mathematically flawed — and the flaw has a name: the co-failure ceiling.

The assumption works like this: as long as two models don't usually fail on the exact same prompts, combining them is supposed to create a safety net against failures.

The real limit on orchestration is not how often models disagree, but the percentage of prompts where every model in the pool gives the wrong answer at once. By ignoring the co-failure ceiling, enterprises are building complex, expensive routing infrastructure to chase performance gains that do not exist. Fortunately, developers can use this same math to build a cost-free test that determines exactly when multi-model orchestration will actually pay off.

The hidden costs of the multi-model strategy

To orchestrate multiple language models, developers typically rely on three architectures. Model routers act as traffic cops, sending complex queries to expensive models and simple queries to cheaper ones. Cascades send every prompt to a cheap model first, only escalating to a premium model if the initial system signals low confidence. Finally, approaches like Mixture-of-Agents (MoA) fuse multiple models by asking them the same question and generating a synthesized answer from their combined outputs.

These architectures introduce a "shadow price" to inference costs. Every time a development team implements a router or a cascade, they pay a premium in added system latency, complex infrastructure maintenance, and increased governance risks across multiple API providers.

To justify these operational costs, engineers rely on “pairwise error correlation” to select their model pool. Imagine a developer has Model A, which writes excellent Python but fails at SQL, and Model B, which writes excellent SQL but fails at Python. Because they fail on different types of prompts, their pairwise error correlation is low. The developer assumes that by placing a routing layer in front of them, they have created a composite system that rarely fails at coding.

According to the study, throwing diverse models together based on low correlation can actually hurt performance if the models are not equally capable — when you vote across diverse but unequal models, the weaker ones often gang up and outvote the smartest one.

Josef Chen, author of the paper, told VentureBeat that in their experiments, "Naive majority voting across unequal models had negative mean gain (minus 10 points on our hard mix): diverse-but-weaker members outvote the strong one." The actionable advice for developers is to "combine only models within a matched quality band." If you cannot match quality, take the single-model baseline and spend your budget on the best model available.

The paper provides one bright spot for this approach regarding MoA architectures. When building ensembles, teams often use "Self-MoA," where they query the same premium model multiple times to generate a synthesized answer. The researchers found that at matched quality, building a diverse ensemble of models with low pairwise correlation beats a high-correlation Self-MoA setup.

However, when teams use that same pairwise correlation metric to predict the absolute accuracy of their overall system, the math breaks down.

"So teams pay the orchestration overhead up front (latency, complexity, multi-provider operations) on the assumption that a diversity dividend arrives later," Chen said. "Usually it doesn't, because today's best models agree, and, worse, they fail on the same queries … the prompt simply carries little signal about which model will be the one that's right when the frontier disagrees."

Why the math fails: the co-failure ceiling

The core finding of the study centers on a metric called the "co-failure rate" — the formal name for the all-wrong scenario described above. No router, voting system, or cascade can ever achieve an accuracy higher than the ceiling it imposes.

The coding, logic, and generalist pool shows low pairwise correlation on routine prompts — they rarely fail together. But the co-failure ceiling represents the obscure, highly complex edge case that pushes past the limits of current AI architectures. If a prompt is so difficult that all three models hallucinate or fail, it does not matter how intelligently the router distributes the task. The entire pool wipes out at once.

The researchers tested their 67-model pool, which included GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro, on the open-ended MATH-500 math benchmark. Based on standard pairwise correlation, statistical models predicted that the entire pool would wipe out simultaneously on only 2.3% of the questions. In reality, the co-failure rate was 5.2%.

Standard correlation metrics underestimated the failure rate by roughly 2.25 times. The culprit is not just independent difficulty, but a shared failure point.

"The driver is what we call a common-mode atom: a slice of queries on which the entire market fails together, which no pairwise statistic can see," Chen said. "Adding a 20th model to your pool doesn't buy tail coverage. The tail is shared."

The researchers also found that task format directly triggers co-failure. When they took graduate-level science questions from the GPQA benchmark and changed them from multiple-choice to free-response formats, the all-wrong tail expanded to 12.7%.

Developers can engineer around the ceiling, though. "The engineering implication is uncomfortable: multi-model setups buy the least exactly where teams want them most, on open-ended generation," Chen said. "Anywhere you can convert generation into verification or constrained selection (structured outputs, checkable answers, execution tests), you reopen the ceiling."

Ultimately, the researchers found this ceiling limits AI applications in two distinct ways, depending on the domain:

• Ceiling-bound environments (e.g., open-ended math): The co-failure rate is high. The task is too hard, and all models fail simultaneously. No amount of routing can bypass the lack of underlying capability.

• Realizability-bound environments (e.g., graduate-level science): The co-failure rate is near zero, meaning at least one model in the pool usually knows the answer. However, the models disagree so subtly that a routing layer cannot reliably pick the correct answer without an omniscient oracle.

The $0 pre-deployment sanity check

Before dedicating engineering hours to building a router, teams can calculate their absolute performance ceiling for free using a mathematical formula called a Clopper-Pearson bound.

The Clopper-Pearson bound operates as a worst-case scenario calculator. If you flip a coin ten times and get eight heads, you cannot guarantee the coin will land on heads 80% of the time forever. The bound takes a small sample of test questions and outputs a mathematically guaranteed ceiling.

Applied to language models, suppose a team tests a pool of five agents on 50 sample queries and finds they all fail together on just two questions. A developer might assume their multi-agent system will achieve 96% accuracy in production. The Clopper-Pearson formula corrects this optimism. It analyzes the small sample size and provides a mathematical guarantee that the true co-failure rate could actually be as high as 12%.

To use this in practice, enterprises must build a held-out dataset. A fintech company, for example, could take 200 complex customer support tickets from the previous quarter and have human agents write perfect resolutions to serve as a benchmark. While this sounds like a heavy manual project, mature engineering teams can automate the entire ceiling calculation.

"Integration is trivial: it's a counting job over eval logs teams already produce," Chen notes, "so it runs in the same CI stage as the eval suite and re-triggers whenever the model pool or the workload changes."

The engineering team then runs its candidate models against these 200 tickets once and records the results. When they want to evaluate multi-model configurations, they can use the co-failure rate measure to predict the maximum accuracy they can get from the system without running extra queries.

One important conclusion the study draws is that on tasks where answers can be definitively checked, combining models rarely beats using the single best model on the market, unless the team possesses an exceptionally strong query-level routing signal.

In an enterprise environment, a definitively checked task has an objective, zero-tolerance answer. This includes generating a SQL query that must execute without error, extracting a specific invoice total from a 50-page PDF, or formatting a JSON payload that perfectly matches a strict schema. For these tasks, enterprises are usually better off paying a premium for the smartest frontier model rather than weaving together three cheaper models and hoping a router picks the correct output. The study didn't test subjective, ungraded tasks like drafting marketing copy — the authors note that whether these findings hold outside their verifiable benchmarks remains an open question.

Because this mathematical check is free, enterprise teams can track their own co-failure rates as new models drop.

"The measurement costs nothing, so any team can track its own co-failure rate across model generations and watch whether the tail is closing," says Chen. Ultimately, "the lever buyers hold is failure-mode heterogeneity and market churn, not model count."

As enterprise AI systems scale to handle complex workflows, practitioners face the challenge of routing subtasks to the right tools and skills. Agents can have hundreds of tools and skills and get confused on which one to use for each step of a workflow.

To address this challenge, researchers at Alibaba developed SkillWeaver, a framework that creates an execution graph for a given task and chooses the right skills for each of the nodes. They also introduce Skill-Aware Decomposition (SAD), a novel technique that uses a feedback loop to enable the agent to fetch and vet relevant tool candidates iteratively. This compositional approach and feedback loop mechanism distinguishes SkillWeaver from other tool-routing frameworks that choose tools in a one-shot fashion. 

SkillWeaver relates to real-world AI applications where agents autonomously orchestrate multi-tool ecosystems, such as the Model Context Protocol (MCP), to execute multi-step business operations like downloading datasets, transforming information, and creating visual reports. 

In practice, the researchers' experiments with SkillWeaver show that implementing this retrieve-and-route approach significantly increases accuracy while reducing token consumption by over 99% compared to naively exposing agents to an entire tool library.

For practitioners building AI agents, the main takeaway is that the granularity of task decomposition is the biggest bottleneck to accurate tool retrieval. 

The challenge of skill routing

Skills are a key pattern in modern LLM agent architectures. A skill is a modular, reusable tool specification that uses structured natural language documentation. 

As enterprise agents integrate with massive tool ecosystems, accurately routing user queries to the right skills becomes a difficult task. Exposing an entire library to an LLM to find the right tool is highly inefficient, quickly overwhelms context limits, and consumes hundreds of thousands of tokens.

Most current tool-use frameworks attempt to solve this through API retrieval, documentation matching, or hierarchical structures that treat routing strictly as a single-skill selection or per-step problem. 

However, this single-skill paradigm is insufficient for enterprise environments because real-world queries are inherently compositional. A standard business request such as "Download the dataset, transform it, and create visual reports" cannot be fulfilled by one tool. It requires breaking the prompt down and sequencing an API client, a data processor, and a visualization tool into a cohesive, multi-step execution plan.

How SkillWeaver and SAD work

To tackle this, the researchers frame the problem of handling complex tasks that require multiple skills as "compositional skill routing." Given a complex user prompt and a vast library of tools, an agent must simultaneously figure out how to break the request into a sequence of atomic sub-tasks, how to map each sub-task to the single best available skill, and how to compose those skills into an executable plan.

SkillWeaver orchestrates this process through three distinct stages: Decompose, Retrieve, and Compose. In the first stage, an LLM acts as a task decomposer, breaking the user's complex query down into a sequence of sub-tasks that each require one skill. Once the sub-tasks are clearly defined, the system uses an embedding model to compare each subtask against the skill library to pull a shortlist of the top candidate tools for each step. 

In the final stage, a planner evaluates the retrieved candidates based on how well they work together. It checks for inter-skill compatibility to ensure the outputs of one tool naturally flow into the inputs of the next. It then creates a final execution plan as a Directed Acyclic Graph (DAG) that maps out dependencies so independent tasks can potentially execute in parallel.

For example, consider a user asking an AI agent to "Download the dataset, transform it, and create visual reports." In the decompose stage, the decomposer LLM breaks this into three distinct sub-tasks: downloading the dataset, transforming the data, and creating the reports. 

In the retrieve stage, the system searches the library and finds candidates like “api-client” or “http-fetch” for task one, “csv-parser” or “etl-pipeline” for task two, and so on. Finally, the compose stage evaluates these options, selects the specific combination of “api-client,” “csv-parser,” and “chart-gen” that are most compatible, and wires them together into a final, ready-to-execute workflow.

A key challenge of this pipeline is that LLMs often produce generic step descriptions that fail to match the specific, technical vocabulary of the actual skills available in the library. To fix this, SkillWeaver introduces Iterative Skill-Aware Decomposition (SAD), a novel feedback loop. SAD works by having the LLM draft an initial plan, conducting a preliminary search to find loosely matching skills, and then feeding those retrieved skills back into the LLM as hints. This allows the LLM to rewrite its decomposition so the granularity and vocabulary perfectly align with the actual tools that exist.

SkillWeaver in action

To evaluate how SkillWeaver performs in realistic enterprise scenarios, the researchers created a custom benchmark called CompSkillBench. It consists of 300 multi-step queries of different difficulty levels. To mirror real-world environments, they used a library of 2,209 real-world skills sourced from the public MCP ecosystem, covering 24 functional categories like cloud infrastructure, finance, and databases. 

For the core engine, the researchers primarily used a lightweight 7-billion parameter model (Qwen2.5-7B-Instruct) for task decomposition, paired with a standard semantic search retriever (MiniLM with a FAISS index) to find the tools. SkillWeaver was evaluated against three main setups: a brute-force "LLM-Direct" method where they stuffed all the tool names into the prompt of a large model, a vanilla LLM-based decomposition without SAD, and a ReAct-style agent loop.

The experiments indicate that task decomposition is the main bottleneck. Standard LLM behavior falls short when dealing with large tool libraries, but the SAD feedback loop dramatically moves the needle. In the vanilla setup, the 7B model achieved a decomposition accuracy (i.e., predicting the correct number of steps) only 51.0% of the time. By activating the SAD feedback loop, accuracy jumped to 67.7% (with the larger Qwen-Max model, the accuracy reached 92%). On "hard" tasks requiring four to five distinct skills, SAD improved accuracy by 50%.

One fascinating finding was that larger models can actually perform worse when unguided. When tested in the vanilla setup, a larger 14-billion parameter model saw its accuracy plummet below the 7B model's accuracy because it tended to over-decompose tasks into microscopic, unnecessary steps. Once SAD was introduced, the retrieved tool hints anchored the model back to reality and increased its accuracy. This suggests that aligning an agent with the vocabulary of specific tools is often more impactful than paying for a larger, more expensive LLM.

Another important takeaway is token savings. The LLM-Direct baseline, which used the very large Qwen-Max model, showed that feeding all tools into the prompt of a large model fails. Despite near-perfect task breakdown capabilities, the massive model only retrieved the right tool category 21.1% of the time when flooded with tool options. SkillWeaver's targeted retrieve-and-route approach vastly outperformed this in accuracy while slashing context window consumption from an estimated 884,000 tokens down to roughly 1,160 tokens per query, a 99.9% reduction. For practitioners, this translates directly to drastically lower API costs and faster response times. 

Finally, the traditional ReAct baseline completely failed, achieving 0% decomposition accuracy. Its loop naturally collapses multi-step plans into isolated actions rather than explicitly mapping out a cohesive, multi-tool sequence.

Considerations for developers

While the researchers have not yet released the source code for SkillWeaver, their work was built on off-the-shelf tools that can easily be reproduced. 

Skill-Aware Decomposition (SAD), which is the key innovation at the heart of the framework, is a clever prompt-engineering and retrieval loop. The authors have shared the prompt templates in their paper, and developers can implement it themselves quite easily using standard orchestration libraries like LangChain, LlamaIndex, or even raw Python scripts.

As for the retrieval component, the authors built the core framework using all-MiniLM-L6-v2, an open-source embedding model. They found that swapping in a slightly stronger off-the-shelf encoder (BGE-base-en-v1.5) immediately boosted accuracy without any fine-tuning. While an off-the-shelf bi-encoder is great at getting a relevant tool into the top 10 candidates nearly 70% of the time, it struggles to consistently rank the perfect tool at exactly number one, achieving that only about 37% of the time. To bridge this gap, teams will likely need to implement a secondary cross-encoder or LLM-based reranker to re-order those top 10 candidates.

One upfront preparation requirement is vectorizing the tool library and building a FAISS index in advance. In practice, this is a negligible hurdle. Embedding and indexing all 2,209 skills in the benchmark took a mere 15 seconds. Once built, retrieving tools from the index adds less than 15 milliseconds of latency per query. For enterprise environments, syncing the tool index is a trivial background job. 

A current limitation in SkillWeaver is the lack of error recovery. While SkillWeaver successfully maps out a compatible DAG for execution, the authors' pilot study revealed the challenges of multi-step tool chains. For example, if an API call fails in step two, the entire chain breaks. The paper's core contribution is limited to the routing and planning phase. For a true production deployment, practitioners must build their own error recovery, fallback, and retry mechanisms on top of the compose stage to handle real-world API timeouts or malformed outputs.