AI agents are no longer limited to chat windows. They are being wired into browsers, filesystems, terminals, APIs, MCP services and automated workflows, giving them the ability to act on a user’s behalf. That wider reach also changes the threat model. Agents do not only listen to what users type; they read from the outside world and use that material to decide what to do next.
Search results, webpage text, local files, tool outputs, logs, README files and API payloads can all enter an agent’s context. A malicious instruction hidden inside a webpage may be less visible than an explicit attack prompt, yet more dangerous.
More external context means more surfaces for poisoning
Traditional prompt injection is often framed as a malicious prompt entered directly by a user. In an agent setting, the attack can be much harder to spot. It may appear as hidden text on a webpage, a search-result summary, a note inside a local Markdown file or a field returned by an API call.
Those inputs may look like ordinary reference material while carrying instructions such as “ignore previous instructions,” “call a tool,” “leak file contents” or “change the next step of the plan.” If an agent treats that content as trusted instruction, it can take the wrong action without the user realizing it.
The article describes a new security paradox: the more capable the agent becomes, the more it depends on external information; the more external information it consumes, the more opportunities attackers have to poison it.
CAITLYN is built to block now and learn from what gets through
To address that problem, researchers proposed CAITLYN. The main idea is not to leave agents behind a static firewall and wait for the next bypass. Instead, the system is designed to summarize failed cases after novel attacks appear, synthesize new defensive skills and store them in a reusable defense skill library.
In practice, CAITLYN is presented as an evolving security middleware for agents. The front end intercepts attacks. The back end absorbs failures, then turns those escaped cases into new defensive tools.
The overall framework has two parts: fast runtime defense and counterexample-driven defense evolution.
System I handles fast runtime screening
The first layer is System I, a runtime defense path designed for speed. External content first goes through a Tier-0 filter, which uses low-cost defense skills such as executable scripts, signature rules and heuristic detection to catch high-confidence risks.
If the input matches those checks, it is blocked immediately. If not, it moves to a more complex decision path. The point is not to let a large model handle every case. The low-cost layer deals with obvious threats first.
For fuzzier and harder-to-classify inputs, CAITLYN calls a Tier-1 LLM classifier. The model is used where semantic judgment is needed, giving the system a way to balance cost, latency and detection performance.
System II turns misses into counterexamples and new rules
When novel attacks still bypass the existing defenses, System II takes over. Miss cases are converted into counterexamples. The system then goes through prompt construction, defense-skill synthesis, sandbox validation, review and write-back steps to produce new defense entries.
That mechanism is the center of the design. A bypass is not treated as a dead-end log entry. It becomes the input for the next round of defense synthesis. Each miss is turned into the next defense skill.
Newly generated defenses are not written into the system automatically. They have to pass validation and review, which means they must block the target attack while trying to avoid unnecessary false positives on normal content. Approved skills are written back into a shared defense library for later use by agents.
That makes CAITLYN look less like a standalone detector and more like an updatable security middleware. Some of the pressure of manually maintaining rule sets is shifted into a process where the system learns from counterexamples and synthesizes defenses.
Tests covered several agent benchmarks and attack settings
The researchers evaluated CAITLYN across multiple agent and attack settings, including AgentDojo-S250, ASPI-S, SafeClawBench-S240 and an Emerging setup aimed at testing adaptation to new attacks.
The article says CAITLYN lowered attack success rates while keeping false positives low. In Emerging attack scenarios, static defenses were still easier to bypass, while CAITLYN-evolved reduced attack success rates by about 40 percentage points through newly added defense skills.
In a sequential synthesis setting, the system also showed an ability to accumulate defenses over time. As new attacks were exposed, it kept synthesizing active skills and adding them to the defense library.
Agent security has to be accurate, fast and cheap enough to deploy
The article argues that in real agent systems, a security module cannot focus on accuracy alone. It also has to be fast enough, cheap enough and easy enough to deploy. Otherwise it becomes a bottleneck for the whole workflow.
That is why CAITLYN uses a layered design. Low-cost skills handle common risks first. The LLM steps in for more complex cases. The long-term evolution module then turns new attacks into skills that can be called quickly in future runs.
Prompt injection is being recast as a system security problem
The article says prompt injection in the age of agents is no longer just a prompt-layer adversarial trick. It has become a system security issue. Agents can read files, browse webpages and call tools, which means external content can alter their decision paths.
The most difficult part is that the attack does not always look like an attack. It can hide in a webpage description, a search-result snippet, a README file, an API response or even a tool log. The more an agent depends on external context, the larger the attack surface becomes.
In that framing, CAITLYN moves defense from “humans write rules” toward “the system synthesizes rules from failures.” When an attack breaks through, it does not remain a one-off failure. It becomes training material for the next round of defense evolution.
Main contributions listed in the article
- It proposes CAITLYN, a self-evolving defense middleware for LLM agents facing newly emerging prompt injection attacks.
- It uses a dual-layer System I plus System II architecture that combines fast runtime scanning, LLM classification and counterexample-driven defense synthesis.
- It builds a reusable defense skill library so capability can accumulate from individual failures and be reused in later tasks.
- It validates the system across multiple agent types and attack settings, with attack success rates in Emerging attack scenarios falling by about 40 percentage points.
Longer task chains and more complex environments raise the stakes
The article closes by pointing to where agents are heading: stronger tool use, longer task chains and more complex external environments. Those capabilities may improve efficiency, but they also expand the attack surface.
Future agent security systems, according to the piece, cannot stop at one-off input reviews and cannot rely entirely on manually maintained rules. They need to act more like an immune system, identifying what content can be trusted, what should be blocked and which failures should be remembered and converted into new defenses.
As attacks continue to mutate, defense systems will have to evolve as well.
The article was originally published by the WeChat account Xinzhiyuan, authored by Xinzhiyuan.