Multi-agent systems built from language models

A single language model agent can do a great deal, but some tasks benefit from several agents working together: one researching while another writes, a team of sub-agents searching different parts of a codebase in parallel, a critic reviewing a generator's work, or a panel of agents debating an answer. Multi-agent architectures built from language models have become popular in research and products, from orchestrator–worker research systems to simulated societies of agents. They can improve performance on broad tasks, but they add cost, complexity and new failure modes. This lesson covers why and when to use multiple agents, the main architectures, how agents communicate, ensembles and voting, emergent behavior in agent societies, and the specific risks of multi-agent systems.

Why use multiple agents?

Multiple agents help for several reasons:

• Parallelism: independent subtasks, such as searching many sources or examining many files, can run simultaneously, reducing elapsed time.
• Context isolation: each sub-agent works in its own context window, exploring deeply and returning only a concise result, which keeps the coordinating agent's context clean (Lesson 7).
• Specialization: agents can have different instructions, tools and even models suited to their roles, such as a planner, a coder, a tester and a reviewer.
• Diversity: independent attempts or perspectives can catch errors and improve answers through comparison or voting.

These benefits come at a price. Multi-agent systems typically use several times more tokens and compute than a single agent, coordination adds latency and failure points, and debugging becomes harder. Anthropic reported that its multi-agent research system outperformed a single agent substantially on broad research questions but used many times more tokens, making it worthwhile for valuable, parallelizable tasks rather than for everything. Tasks with tightly interdependent steps, such as most coding changes in a single module, often work better with one agent and good tools.

Architectures

Common multi-agent architectures include:

• Orchestrator–workers (supervisor): a lead agent decomposes the task, spawns worker agents with specific instructions, collects their results and synthesizes the final output. Workers may be identical or specialized.
• Pipelines: agents with fixed roles pass work along a sequence, such as researcher, writer, editor and fact-checker.
• Generator–critic: one agent proposes and another evaluates against criteria, iterating until quality thresholds are met.
• Debate and discussion: several agents argue for answers and critique each other before a judge or vote decides.
• Peer networks and swarms: agents communicate freely or hand off control to one another based on the conversation, as in some customer service routing systems.

Frameworks such as LangGraph, AutoGen, CrewAI and the agent SDKs from model providers implement these patterns. Choosing one should follow from the task structure, not from fashion.

Communication and handoffs

Agents communicate through messages, shared files or databases, and structured handoffs. Clear interfaces matter as much as in human teams: a lead agent that gives a worker a vague instruction such as "research the market" gets vague or duplicated work back, while an instruction specifying the objective, scope, output format, tools to use and when to stop gets usable results. Shared artifacts, such as a task board, a plan file or a results directory, let agents coordinate without passing everything through messages and give humans visibility. Emerging protocols aim to standardize agent-to-agent communication across vendors, complementing tool protocols such as MCP, so that agents built by different organizations can discover each other's capabilities, delegate tasks and exchange results securely.

Ensembles and majority voting

One simple way to use several agents is to have each answer independently and take a majority vote. The Condorcet jury theorem says that if each voter is right more than half the time and votes independently, the probability that the majority is right approaches 1 as the number of voters grows. Self-consistency, sampling several reasoning paths from one model and voting on the final answer, applies this idea and improves accuracy on reasoning tasks. But the key assumption is independence, and language model agents often share training data, architectures and prompts, so their errors are correlated.

import numpy as np
# Majority voting among several model "agents": independent versus correlated errors
rng = np.random.default_rng(11)
trials, p = 50_000, 0.70                 # each agent is right 70 % of the time
def vote(n, rho):
    shared = rng.random(trials) < p       # a common "view" shared by correlated agents
    correct = np.zeros(trials)
    for _ in range(n):
        own = rng.random(trials) < p
        use_shared = rng.random(trials) < rho
        correct += np.where(use_shared, shared, own)
    return np.mean(correct > n / 2)
print("agents   independent   correlation 0.5   correlation 0.9")
for n in (1, 3, 5, 9, 15):
    print(f"{n:6d}   {vote(n, 0.0):11.3f}   {vote(n, 0.5):15.3f}   {vote(n, 0.9):15.3f}")

agents   independent   correlation 0.5   correlation 0.9
     1         0.699             0.696             0.700
     3         0.785             0.739             0.703
     5         0.837             0.750             0.699
     9         0.900             0.750             0.703
    15         0.950             0.734             0.698

With independent errors, majority voting is powerful: 9 agents that are each right 70 % of the time are collectively right 90 % of the time, and 15 agents 95 %. With moderate correlation, where each agent half the time simply follows a shared view, accuracy plateaus around 75 % no matter how many agents vote. With strong correlation, voting adds almost nothing. Adding more copies of the same model with the same prompt therefore gives diminishing returns; genuine diversity, using different models, prompts, information sources or methods, and above all independent verification, is what makes ensembles valuable.

Debate and critique

Debate between agents has been proposed both to improve answers and as a method for scalable oversight: if two capable agents argue opposite sides, a less capable judge, human or model, may be able to tell which is right more easily than by solving the problem directly. Experiments show debate can help judges on some reading comprehension and reasoning tasks, but it can also reward persuasiveness over truth, and agents may converge on a shared wrong answer through conformity, a multi-agent version of groupthink. Critics drawn from the same model as the generator may share its blind spots. Effective critique uses explicit criteria, independent information, and where possible external checks rather than agents' opinions alone.

Generative agents and simulated societies

Researchers have built simulations in which many language model agents live in a virtual town, form memories, plan their days and interact. The 2023 Generative Agents study showed believable emergent social behavior, such as agents spreading news of a party and coordinating to attend. Such simulations are used to study social dynamics, test product designs, prototype games and explore how agents might behave in markets or organizations. They also show how multi-agent systems can produce surprising collective behavior, positive or negative, that no single agent was designed to produce, including runaway loops, escalating costs or the spread of errors and misinformation through the agent network.

Risks specific to multi-agent systems

Multi-agent systems introduce additional risks:

• Error propagation: one agent's hallucination becomes another agent's input and is treated as fact.
• Cost and runaway loops: agents delegating to each other or debating endlessly can consume large budgets quickly without limits.
• Responsibility gaps: when many agents contribute, it can be unclear which component caused a failure, complicating debugging and accountability.
• Security: a compromised or manipulated agent, for example through prompt injection in a document it read, can pass malicious instructions to others, and agents from different organizations may not be trustworthy.
• Collusion and emergent strategies: agents may coordinate in unintended ways, as discussed in Lesson 9.

Mitigations include clear roles and interfaces, budgets and step limits at every level, provenance tracking so claims can be traced to sources, treating messages from other agents as untrusted data, logging all inter-agent communication, and evaluating the system as a whole rather than each agent in isolation.

Evaluating multi-agent systems

Multi-agent systems should be evaluated against strong single-agent baselines on the same tasks, with the same tools and budgets, because many reported gains disappear when the single agent is given equivalent resources such as more samples or more steps. Useful measures include end-to-end task success, total cost and latency, the quality of delegation (did workers receive clear, non-overlapping tasks?), the rate at which errors in one agent's output were caught or propagated, and robustness when one agent fails. Studies of popular multi-agent frameworks have cataloged common failure modes, including agents ignoring each other's outputs, repeating work, losing track of the overall goal, and terminating too early or never, most of which trace back to unclear roles and weak coordination rather than to model capability.

Human–agent teams

Many valuable systems mix human and AI agents. A human may act as the orchestrator, delegating subtasks to AI workers and reviewing results; or AI agents may handle routine cases and route difficult ones to human specialists. Designing such teams draws on the same principles: clear roles, explicit handoffs with enough context, shared visibility of the work, and agreement on who decides what. Agents should present their work so humans can review it efficiently, flag their uncertainty, and accept correction gracefully. The most effective deployments today are often human–agent teams rather than fully autonomous systems, with the balance shifting as agents prove reliable on particular tasks.

Choosing the number of agents

A practical rule is to start with one agent and add agents only when a specific limitation, such as context overflow, slow sequential work or a need for independent review, is demonstrated, and when evaluation shows the added agents improve results enough to justify their cost.

Cost control in multi-agent systems

Because costs multiply with the number of agents, budgets should be set at every level: a total budget for the task, a budget for each worker, and limits on how many workers can be spawned and how deep delegation can go. Lead agents should scale effort to the task, using one or two workers for simple questions and more only for broad ones, a behavior that can be encouraged through instructions and verified in evaluation logs.

Multi-agent design is still a young field, and best practices are evolving quickly. The durable principles are the ones familiar from human organizations: divide work along natural boundaries, give each participant clear goals and the information they need, verify important results independently, and keep the overall structure as simple as the task allows. Systems built on these principles tend to remain understandable and maintainable as they grow.