What a 49.1% attack rate does not tell you

The number

49.1 percent. That is how often a new kind of attack worked on an AI agent with no defences turned on.

The attack is called agent data injection. An attacker puts quote-like characters inside a field they control, such as a comment. The model then misreads where one field stops and the next starts. It treats the attacker’s text as an official field, like the author name. No order is given. The agent just believes the wrong thing.

The rate comes from AgentDojo, a public test bench with pretend email, Slack, banking and travel tools. Our full write-up is here: Fake quote marks fool AI agents half the time.

Where it comes from

The paper is “Agent Data Injection Attacks are Realistic Threats to AI Agents”, from Seoul National University, Largosoft and the University of Illinois Urbana-Champaign. It went up as a preprint in July 2026.

The agent test used 96 user tasks and 108 new attacks, against seven defences. The bench already held 935 older prompt injection attacks for comparison. A separate model study used 157 test cases across six models and seven kinds of data.

We score it 6 out of 10. No journal or conference has reviewed it. It gains points for open code and for three institutions doing the work. It gains most for one thing. The team built working attacks on shipping products and told the makers first. OpenAI, Google and Anthropic confirmed the reports.

What it does not mean

It does not mean half the attacks on your agent will work. 49.1 percent is a per-try rate. It was measured on 108 attacks the authors wrote themselves, inside a simulation. Nobody counted attacks in the wild here. It tells you how often the trick works when someone tries it, not how often people try.

It does not mean the attack is hundreds of times worse than the old one. You can divide 49.1 by 0.2 and get a big number. Do not use it. The two figures come from different sets of attacks, of different sizes, made by different people. The pair shows that today’s guards miss one shape and catch the other. It is not a measure of danger.

It does not mean your agent scores 49.1 percent. That figure is for an agent with nothing switched on, running one model. A sandbox brought the same attack down to 22.2 percent. One strict defence stopped every attack, but then normal task completion fell from 86.5 percent to 36.5 percent. Your own number depends on the model you run and on what you already have switched on.

It does not mean old prompt injection is solved. The 0.2 percent applies to those 935 stored attacks, against that agent, on that day. It shows the industry got good at the attack it had studied. A low score on old attacks says nothing about new ones.

It does not mean a coin flip is a tolerable risk. People hear “half the time” and relax. An attacker who can post one comment can post twenty. A guard that works one try in two is not a guard.

What it does mean

The guards were built for one shape of attack. This is a different shape. The old worry was a hidden order inside a web page, and filters now catch that language. This attack gives no order. It changes what the agent thinks the data is.

The defence results show it plainly. An input filter caught none of the 108 new attacks. The same filter caught 326 of the 935 old ones. An output checker flagged only 14 of the new attacks, and just one of those flags named the real cause.

One more result is worth keeping. The fake marks do not even have to look right. On one model, wrong marks such as a curly quote or a dollar sign still worked 35.8 to 43.3 percent of the time. A normal program would reject those. A model guesses.

Why it matters to you

This matters if your agent reads anything a stranger can write. A support inbox. A public issue tracker. A shared drive. A web page.

The real lesson is not a rate. Your agent cannot tell data it should trust from data it should not. Everything arrives as one stream of text. THAT IS THE HOLE. A field name inside it is not proof of who wrote it.

There is a cheap step that moved the number. Add a short random code to each field name, so the field is “author_f7x9k2” and not “author”. An attacker cannot guess the code. In the full agent test this cut the attack from 49.1 percent to 28.7 percent, and normal work still finished 83.3 percent of the time. On single models it cut a 31 to 43 percent rate down to 0 to 3 percent. That is not a full fix, but it is cheap.

Do this today

Find one place where your agent trusts a name it read inside a message. An author, a sender, an approver. Make the agent check that name with a separate call to the service before it acts. If you cannot make that call, the agent should not act on the name.

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.