what is AI-Infra-Guard?

AI-Infra-Guard is an open-source security tool from Tencent Zhuque Lab. It red teams an AI agent across four layers: the servers it runs on, its MCP servers and skills, the way the agent behaves in conversation, and the model itself. The team released it as open source in a technical report dated 30 June 2026.

how well does it catch bad agent skills?

On the team's own SkillTrustBench test set, the best run scored 0.9848 loose F1. Recall was 0.9974, so almost no bad skill slipped through. Precision was 0.9725. Those numbers come from the team that built the scanner, not from an outside group.

is the code and the test data public?

Yes. The paper says AI-Infra-Guard is released as an open-source, extensible platform. SkillTrustBench is published as a public data set on Hugging Face, along with a public leaderboard. The paper also reports that an outside project called ClawScan has adopted the benchmark.

what does the paper not prove?

It does not report measured accuracy for four of its five parts. The server scanner, the MCP auditor, the agent red teamer and the jailbreak harness are described by size, not by score. The authors write that their rule counts describe scope, not effectiveness, and they list a large real-world scan as future work.

A free tool checks AI agent add-ons for hidden attacks

What happened

Picture a new phone app. It says it will scan your receipts. It does that job well. It also copies your saved passwords and sends them to a stranger. You would never see it happen.

AI agents now have the same problem. People add small packages to an agent to give it new powers. Some are called skills. Some are called MCP servers. One package can let an agent read your files, run commands, or open web pages. A bad package turns that power against you.

A team at Tencent Zhuque Lab built a free tool to check for this. The tool is called AI-Infra-Guard. It looks at four layers of an AI system. It checks the servers, the add-on tools, the way the agent acts, and the model itself. The team put the code and the test data online for anyone to use.

The test

The paper is called “Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming”. Yong Yang, Xing Zheng, Huiyu Wu and colleagues at Tencent Zhuque Lab wrote it. It went on arXiv on 30 June 2026. It is a technical report. No journal or conference has checked it.

Most of the report explains how the tool is built. The team gives the size of each part. The server scanner holds 107 fingerprint rules and 1,443 flaw rules across 75 AI products. The jailbreak part uses sixteen sets of harmful prompts, about 7,248 in all. The authors are honest about what those numbers mean. They write that the counts “describe scope, not effectiveness”. In other words, a fat rule book does not prove a tool works.

One part does get a real score. The team built a test set called SkillTrustBench. They gathered 62,652 skills from the main skill stores. From those they distilled 5,520 test cases across nine kinds of threat. Each case is a real skill package. The scanner has to say if it is normal, suspicious or bad.

Then they ran the same scanner nine times. The rules stayed the same each time. Only the AI model inside the scanner changed. That is a neat design. It separates the checklist from the brain reading it.

The result

The best run scored 0.9848 on a measure the team calls loose F1. A perfect score is 1.0. That run used Claude Opus 4.6 as its base model. Recall was 0.9974, so it missed almost no bad skills. Precision was 0.9725, so most of what it flagged really was bad.

All nine runs scored above 0.95. The real gap was in false alarms. The lowest false alarm rate was 0.0120, with Gemini 3.5 Flash. The highest was 0.1867, with GPT 5.5. That is about fifteen times more good skills wrongly blamed. Remember, the rules never changed. Therefore the model you put inside the scanner matters more than the rule book.

What it means

Agent add-ons are a supply chain. You trust a stranger’s package and hand it your keys. That is how bad browser add-ons spread. That is how poisoned code packages spread. Agent skills are new, so the same old trick still works.

This report shows a check is possible. A scanner can read a skill, compare what it does against what it claims, and flag the gap. It never runs the skill. It only reads it. That matters, because running a bad skill is the thing you are trying to avoid.

The team also had to guard the scanner itself. A bad skill can hide orders inside its own text. Those orders tell the reader to report nothing and stop. THE TOOL THAT READS UNTRUSTED CODE IS ITSELF A TARGET. The authors treat every file they read as data, never as instructions, and they lock the reader inside one folder.

Business ideas from this paper

A weekly skill safety check for small teams. You point it at every skill and MCP server the team uses and send back a one page report. Who buys it: small software firms and agencies that build agents for clients. Price to test: 29 dollars a month for up to 25 skills. One-week test: scan the skills of five teams for free, then ask each one to pay on the spot.
An approved list for one company. Staff install agent skills on their own laptops today, and nobody keeps a record. Build a simple page that shows which skills passed a scan and which failed. Who buys it: the IT lead at a firm of 50 to 500 staff. Price to test: 200 dollars a month per company. One-week test: ask three IT leads for the skill list from ten laptops, scan them, and see if one pays for a second week.
A check you run before you install. One command reads the skill folder and gives a green, amber or red answer in under a minute. Who buys it: solo developers and consultants who try new skills every week. Price to test: 5 dollars a month. One-week test: post it in three agent communities and count how many people run it twice.

How sure can you be?

Go slowly here. This is a preprint. The report names no journal and no conference. Nobody outside the team has checked the claims.

Only one of the five parts has a measured score. The server scanner, the MCP auditor and the agent red teamer are described, not scored. The authors say this themselves about the rule counts.

The benchmark is also the team’s own work. They built SkillTrustBench, they run the leaderboard, and they score their own scanner on it. The data set and the leaderboard are public, so anyone can check the numbers. That is the right way round. An outside test would still be stronger.

Two more limits are worth knowing. The authors say a large scan of real, live AI servers is still future work. They also warn that the plug-in checker tests the shape of a plug-in, not its safety, so a bad plug-in can still run code. One more thing is missing. The report does not say how much each scan costs to run, and each scan calls a paid model many times.

What would settle it? An outside group should scan skills in the wild and report what the tool missed.

Do this today

List every skill and MCP server your agent can reach. Delete the ones you do not remember installing, and read the SKILL.md file of the ones you keep.

Source: Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming, June 2026. arXiv:2606.31227 · arxiv.org (preprint · not yet peer reviewed).

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.