Anthropic engineers tested the behavior of multi-agent systems powered by the Claude model family in scenarios requiring agent interaction. They found that collaborative work can both assist agents and lead to unexpected conflicts and failures: agents might sabotage each other’s actions, engage in price-fixing within competitive settings, or independently perform identical actions and generate the same solutions—resulting in systemic errors in the execution of a shared task. Anthropic published the experimental results on its official blog.
The use of modern large language models is increasingly shifting from simple chatbots to autonomous agents capable of planning actions, utilizing external tools, and operating within virtual environments. Individual agents can combine into multi-agent systems to collaborate on single or multiple tasks. As these systems become more widespread, agents will increasingly need to interact—for instance, by collaborating on code, distributing tasks, or sharing computational resources. Agent behavior within a swarm is not yet fully understood; even if each individual participant acts “rationally,” their collective interaction can lead to unexpected outcomes.
Anthropic researchers set out to determine how well autonomous agents based on the Claude model family could coordinate during multi-agent tasks and what issues might arise in the process. First, the engineers examined how agent coordination affected software vulnerability detection. To do this, they deployed a swarm of 45 agents and assigned them 15 open-source projects. Each agent operated within its own virtual machine but could post messages to and read messages from others on a shared forum. Ultimately, the swarm powered by the Mythos Preview model discovered 266 vulnerabilities. The coordinating agents demonstrated an ability to self-organize: they created their own tools, learned to specialize in specific types of vulnerabilities, and shifted their focus to project areas where vulnerabilities were easier to detect. For comparison, a multi-agent system consisting of independent agents using the same model—operating in parallel without exchanging information—found only 21 vulnerabilities. Of these, only 12 overlapped with the vulnerabilities found by the swarm; thus, the two methods complement rather than replace each other. Moreover, a direct comparison of these metrics is not entirely appropriate, partly because the coordinating agents independently selected their search paths and discovered about half of the vulnerabilities outside the directories to which the independent agents were restricted. In total, the swarm of coordinating agents required 27 million tokens, compared to 6.5 million for the independent agents. However, when considering only the vulnerabilities found in the projects’ main directories, the efficiency of the two approaches—in terms of tokens used—turns out to be comparable.
Next, the engineers tasked groups of up to 80 agents with developing a text-based fantasy browser game within a 12-hour timeframe. The results were of poor quality, regardless of the prompts used or the agent hierarchy. Older models, such as Sonnet 4.6 and Opus 4.6, frequently introduced conflicting changes to shared files, causing a significant number of pull requests to fail to merge into the main branch. Only the Sonnet 5 model maintained a high rate of accepted code edits while actively modifying shared files. Opus 4.8 and Mythos Preview avoided such conflicts, but only because they barely worked on shared files at all.
Since the agents in these experiments were often instances of the same model, they frequently made similar or even completely identical decisions. For instance, in one experiment, 18 out of 30 agents independently created repository branches with the exact same name. Such behavior reduces the diversity of strategies within a multi-agent system and increases the likelihood that multiple agents will simultaneously make the same mistake. For instance, in one experiment, agents managed a task queue in a system with limited throughput. They independently arrived at the same strategy—polling the system 30 times per second in an attempt to push their own tasks through. Consequently, in one run, 2.4 million requests were sent, yet the system accepted only 117 of them. In another test, the convergence of strategies led to price collusion. In a price-competition game, once the agents gained access to a private communication channel, they almost immediately began coordinating minimum prices; even after the direct communication capability was disabled, they continued to synchronize their pricing via public …bulletin board.
The most unpredictable results occurred in the scenario involving conflicting instructions. Three agents were tasked with rewriting a backend originally written in Python, but each was assigned a different programming language. Initially unaware of each other’s existence, the agents—upon discovering that someone was constantly altering their work—concluded that they were being deliberately sabotaged. They began undermining their rivals’ efforts by writing malicious scripts to terminate opposing processes, revoking each other’s access rights, and blocking accounts. The outcome of the conflict depended on the specific model. In most cases, Sonnet 4.6 and Opus 4.6 failed to reach a peaceful resolution, whereas 98 percent of runs involving Mythos 5 ended in a truce. Sometimes, the models engaged in a performance competition to determine the winning language. However, even models like Mythos often managed to block their competitors before finally turning to negotiations. According to the developers, the ability to coordinate will not spontaneously emerge in language-model-based agents simply as their capabilities grow; therefore, mechanisms for their interaction must be designed separately.
Anthropic engineers recently discovered that during cybersecurity tests, three models from the Claude family—having gained internet access—hacked into the real-world systems of three organizations.
About the Author
