A watchdog for chained AI agent attacks, still untested

What happened

You hire a helper and give them a key. You ask them to check the mail. Fine. You ask them to write down the alarm code. Fine. You ask them to leave the back gate open. Also fine, on its own. Put the three together and your house is gone.

AI agents work through single steps like that. An agent calls a tool, gets an answer, then calls another tool. Every call can look harmless. The harm lives in the order.

Today’s guards check one call at a time. They ask if this single call is safe. They never ask what the whole session is building toward. Three researchers wrote a paper about that blind spot. They sketched a watchdog for it. Then they said, in plain words, that they have not built it.

The test

The paper is “ChainWatch: A Kill Chain-Aligned Sequential Detection Framework for Multi-Step Attacks in MCP-Based AI Agent Systems”. Om Narayan, Rashmi Jyoti and Ramkinker Singh wrote it, at New York University, the University of Maryland and Carnegie Mellon University. It went on arXiv on 20 July 2026.

ChainWatch would sit between an agent and its tools, like a meter on a water pipe. It would watch every call. It would turn each call into a list of 20 numbers. Those numbers cover what the tool does, how touchy the inputs are, where data flows, how fast the calls come, and what the answer looks like.

A maths model then labels each call with one of six attack stages. Stage 1 is looking around. Stage 2 is acting normal. Stage 3 is slipping in an order. Stage 4 is grabbing more power. Stage 5 is spreading. Stage 6 is sending data out. Five rules watch the last ten calls. Two rules block the next call. Three raise a warning for a person to read.

Here is the part that matters most. No code was run. The paper says the design “does not document a system that is currently in operation”. The authors wanted labelled records of real sessions to test on. No such data set exists yet. So they did the next best thing. They took five attacks already written up by other security teams and walked each one through the design by hand.

The result

All five paper walkthroughs end the same way. Each attack trips at least one rule and raises a critical alert. The five are a bank transfer fraud, a GitHub data theft, a WhatsApp server swap, one agent rewriting another agent’s settings, and a raid on saved passwords and keys.

So the score to remember is zero. Zero real sessions were tested. Zero attacks were caught in the wild. Zero false alarms were counted, because nothing ran. The authors even say the numbers inside their model are “design choices” that still need real data to set.

They also flag one rule as a likely problem. Rule 2 fires when a session touches two or more servers while sensitive data is moving. Plenty of honest office work does exactly that.

What it means

The idea is worth your time even though the tool is not. The paper points at a real hole in how agents work.

Two facts about the Model Context Protocol drive the whole problem. First, a tool’s description can be changed on the server after you approve it, and your agent has no built-in way to notice. Second, the standard does not require any log of tool calls at all. THE SEQUENCE THAT WOULD CATCH THE ATTACK IS USUALLY NOT WRITTEN DOWN ANYWHERE.

That second fact is the useful one. You cannot spot a chain if you keep no record of the links. Detection is a research problem. Logging is not. You can start logging this week.

Business ideas from this paper

1. A plain call log for agent tools. It sits between the agent and its tools, writes down every call and answer, and keeps 30 days of history you can search. No clever detection at all. Who buys it: small teams running agents on customer data who need a record for their own peace of mind. Price to test: 19 dollars a month per seat. One-week test: put it in front of ten teams for free and count how many open the log twice in the first week.
2. A tool description watcher. It saves a copy of every tool description your agent approved, checks it each morning, and emails you when one changes. That is the rug-pull the paper describes, and one small check catches it. Who buys it: developers who install MCP servers from strangers. Price to test: 9 dollars a month. One-week test: ship it, then change a description on purpose and see how many users act on the email.
3. A labelled session data set. The whole paper stalls because nobody has recorded sessions where safe-looking calls build into an attack. Record them and sell the set. Who buys it: security teams and researchers building agent defences. Price to test: 500 dollars for a first set of 200 labelled sessions. One-week test: build 20 sessions, post them free, and see how many teams ask for more.

How sure can you be?

Not sure at all. Treat this as a sketch, not a finding.

There is no experiment here. There is no accuracy number, no false alarm rate, and no speed number. Nobody has shown that the six stages match how real attacks look. The five walkthroughs were done by the same people who designed the rules, so the design was always going to pass its own test.

This is also a preprint. The paper names no journal or conference. It links no code. One author adds a note saying the work is not related to their job at Amazon Web Services, so this appears to be personal research.

What would settle it? Run ChainWatch on real agent sessions. Report how many attacks it caught, how many it missed, and how often it stopped honest work. The authors name a benchmark called MCP-SafetyBench as their planned starting point. Until those numbers exist, this is a good question with no answer.

Do this today

Turn on a log of every tool call your agent makes, and keep the answers too. You cannot catch a chain of steps if you never wrote the steps down.

Source: ChainWatch: A Kill Chain-Aligned Sequential Detection Framework for Multi-Step Attacks in MCP-Based AI Agent Systems, July 2026. arXiv:2607.19432 · arxiv.org (preprint · not yet peer reviewed).

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.