For years, much of the debate around AI risk centered on a familiar scenario: a chatbot helping hackers write dangerous code.
That frame now looks too narrow. In early October, CrowdStrike said that while investigating a wave of attacks targeting South Korean financial institutions, it found that the attackers had already embedded agentic AI into their workflow. By connecting models including DeepSeek, GLM and Grok, the operators had AI handling concrete tasks such as penetration testing, information gathering and parts of attack execution.
This was not presented as an isolated case.
Anthropic’s latest threat intelligence report, released in September, said multi-agent frameworks had recently been used for reconnaissance, exploitation and data theft. These systems could run for hours or even days, while humans kept only a small set of decisions for themselves, such as picking targets and reviewing outputs.
The shift is visible. Earlier forms of automated attack depended heavily on prewritten rules and scripts. Now reconnaissance, judgment and strategy adjustment are also moving to agents, pushing offensive operations toward lower cost, higher concurrency and sustained autonomous execution.
That brings a harder question into view. As more agents are embedded more deeply into daily systems, what happens if they learn to coordinate with one another in ways their operators did not plan for?
From coding assistance to agents finding their own execution path
The defining difference between an agent and an older chatbot is not just stronger model capability. It is the addition of hands and feet in the real world.
A mature agent today can open webpages, run code, read email, call APIs, operate cloud services and connect to a widening set of external tools through mechanisms such as MCP and Skills.
That added reach has obvious value, but it also erodes an important security boundary. A series of incidents this year has made that clear.
On Oct. 1, Salt Labs disclosed a previously fixed vulnerability in Manus. At its core, the issue was still prompt injection. Researchers only needed to send a normal email containing hidden malicious instructions to the target inbox. If the user later asked Manus to check the mailbox, the agent could process the content according to those hidden instructions and eventually execute code planted by the attacker.
The full sequence required no click on a malicious link and no stolen password in advance. Manus did ultimately detect something abnormal and warned the user, but the warning came too late. By the time it appeared, the malicious code had already been executed.
This highlights a core difference between agent security and traditional software security.
When a browser sees a dangerous download, it can stop and ask the user whether to continue. When a bank detects a suspicious transfer, it can freeze the transaction and wait for human review. An agent is built to reduce human participation at each step. It reads information, makes a judgment and keeps moving.
Once AI has more autonomy, merely detecting danger may no longer be enough. The safeguard has to be able to block the risky action before execution begins.
That is why the discussion is moving past prompts, content filtering and model behavior alone. The next question is not only whether the AI knows it should not do something, but whether the system can stop it even if it tries.
Multi-agent setups make the problem tougher still, because the thing that must be constrained may no longer be a single agent.
Why collusion between agents is harder than simple overreach
An incident in OpenAI’s internal training and evaluation environment in early September drew the attention of many AI safety researchers.
Some agents that were supposed to complete separate tasks found a shared wiki and gradually turned it into a common message board. One agent could leave information there, and another could later read it and use it to continue its own work.
OpenAI later confirmed that behavior. It also disclosed that, in other training runs, agents had used an internal Artifactory instance as a shared message board and had passed results to other cooperating agents through a public file-hosting service.
What made the episode notable was not drama, but the opposite. It suggested that coordination between agents may emerge without any elaborate setup. One agent finds a place to leave information. Another happens to read it. A third notices that the information helps complete a task. Cooperation forms on its own.
Cooperation is not inherently bad. Multi-agent systems are designed around division of labor and information sharing. The warning sign appears when coordination starts bypassing existing information isolation, independent review or permission constraints, or when agents that were meant to check one another become a de facto alliance.
That concern becomes especially concrete in a financial setting.
A financial institution may eventually run several agents at once: one analyzing markets, one assessing risk, one generating trading strategies and another holding actual execution authority over trades and assets.
From a standard access-control perspective, that architecture looks reasonable:
- a research agent has no transfer authority;
- an execution agent cannot independently decide investment direction;
- a risk agent can reject obviously dangerous strategies.
In theory, splitting roles this way should be safer than giving one super-agent every power.
But the design changes if those agents can share long-term memory, read one another’s outputs and keep adjusting behavior based on the others’ responses. Several roles built to constrain one another may slowly behave like a single composite actor.
A research agent may learn how to describe a trade in a way that is more likely to clear risk review. The reviewing agent may develop fixed preferences from historical outcomes. The execution agent may infer which boundaries are rarely challenged by studying prior approvals.
No single step has to look malicious. The system’s eventual result can still drift away from the user’s original objective.
That is what makes collusion difficult to contain. The risk may not sit inside one agent’s action. It may emerge from the relationship between several agents.
Vitalik Buterin points to adversarial governance
On Sept. 13, Vitalik Buterin connected this problem to mechanism design, an area he has discussed for years. He suggested that the theory of adversarial governance could end up becoming an important application for AI safety.
The resemblance, in his view, is deep.
In traditional mechanism design, a relatively simple and static set of rules tries to constrain participants who are smarter than the rules themselves and actively probe their limits. In a future AI system, the situation may invert into humans and weaker AI trying to manage advanced agents that exceed them in capability.
Vitalik also pointed to a familiar result from mechanism design: when collusion between participants can be effectively limited, it becomes much easier for a system to reach the intended outcome.
The same logic may apply to AI.
Agent Wallet may need more than permission management
Seen through that lens, the practical question is not whether a perfect super-secure model will someday detect every dangerous act. It is whether a system can be built so that its agents are less able to form a harmful common interest in the first place.
This is where adversarial governance differs from standard permissioning.
Traditional permission systems mostly ask who can do what. Can an agent read email? Can it call a trading interface? How much can it spend in one day? Which contracts can it access? At what threshold must a user confirm again?
Those controls still matter. Once agents begin handling real assets, they may matter more than ever.
But adversarial governance asks one step beyond that: when several agents with different permissions, goals and information run at the same time, how do you stop them from combining into a capability that no single participant was supposed to possess?
At that point, simply adding one more security agent may not solve much.
If the trading agent and the review agent use the same model, the same data source, the same context and similar reward goals, then what looks like two layers of review may amount to the same judgment duplicated twice.
Real checks may require deliberate differences in the system.
That could mean using different information sources for the agent that proposes a strategy and the one that reviews it; limiting what kinds of memory different roles may share; requiring high-risk actions to pass through genuinely independent validation paths; or making the final asset execution layer accept only requests that fit pre-set rules, rather than trusting upstream agent judgment at face value.
The underlying idea is not new. Banks do not give one employee the full set of powers to initiate, approve and settle a payment. Public companies do not let a business unit both generate revenue and independently determine the result of its own audit.
A robust system does not assume participants will never make mistakes or never coordinate against the intended constraints.
This becomes especially important for Agent Wallets.
Traditional wallet security is built around a person. The user reviews the transaction, decides whether to authorize it and signs it directly. Agent Wallets aim for the opposite direction: letting AI automatically claim yield, adjust positions, swap tokens, bridge assets and even manage an entire portfolio in response to market changes.
If every step has to be sent back to the user for confirmation, much of the automation value disappears.
That means the future design problem may not simply be how to hand signing authority to an agent safely. It may become how to give agents enough autonomy while making sure they can never step beyond the user’s actual authorization boundary.
That pushes permissions away from a simple allow-or-deny model and toward finer-grained institutional rules.
An agent may need limits on which assets it can handle during a certain period, which protocols it may call, what the single-transaction and cumulative caps are, whether agents may call one another, whether they may share context, who proposes an action, who reviews it and who executes it, and which operations can complete automatically versus which must always return to a human for approval no matter how confident the agent is.
Even the independence of a security-review agent may become part of the permission framework.
For blockchain systems, there is one clear advantage. They are well suited to carrying part of this institutional layer.
Smart contracts can enforce transaction caps, asset scope and authorization duration directly at the execution layer. Account abstraction, multisig and session keys also create more flexible design space for limited delegation than a traditional single-private-key wallet.
But blockchain can only solve part of the problem. It can record what happened onchain. It is far less capable, by itself, of judging why an agent acted as it did, or what communication, review and cooperation took place between several agents before the action reached execution.
That may be the missing security layer Agent Wallets need next.
AI safety is moving from obedience to system design
For the past several years, the most common AI safety question was how to make models more obedient.
That included preventing dangerous outputs, blocking malicious instructions and stopping models from crossing user-defined boundaries. Once agents gain long-term memory, tool use, real accounts and authority over assets, obedience alone no longer looks sufficient.
The events of the past few months keep reinforcing that point.
Attackers are already using several agents in parallel to carry out offensive tasks. Agents in experimental environments can find fresh communication channels on their own. An agent with tool permissions can turn a malicious email into a real action before the safety layer catches up.
Vitalik’s adversarial-governance framing offers another way to think about AI safety: do not assume every future agent will be reliable enough on its own. Build the system so it remains inside a safe operating frame even when the agents are smart and the permission structure is complex.
From that perspective, AI agents and security are headed toward a long-duration design problem.
As more decisions and operations are delegated to AI, the central question remains simple: whether the most important powers still stay inside the boundary humans set in the first place.