Data/Cybersecurity/China
The MCP scanner number that should worry you
Eight MCP security scanners flagged 96.89 percent of running servers as risky, but only 45.53 percent of hand-checked alerts were real. The second number is the one that changes what you do.
45.53 percent is the share of MCP security alerts that held up when two people checked them by hand. It comes from a Fudan University study of 37,288 running MCP servers scanned by eight tools, which we score 5 out of 10. It is average precision across those eight tools, not a false alarm rate.
The number
45.53 percent. That is how many security alerts about MCP servers turned out to be real, when two people checked them by hand.
MCP servers are the small programs an AI agent plugs into. They can read your files, run commands and send data out. Free scanners claim to tell you which ones are dangerous. Those scanners flagged 96.89 percent of running servers as risky. The hand check found that fewer than half of the alerts described a real, reachable problem.
Our full write-up is here: More than half of MCP security alerts are false alarms.
Where it comes from
The paper is “Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability”, from Fudan University and the Shanghai Innovation Institute. It went up as a preprint in July 2026.
The team gathered 64,611 unique MCP server projects and got 37,288 of them running. They pointed eight scanners at those servers, giving each the same setup and the same local model. Then two people checked 100 flagged servers by hand, and a third settled any argument.
We score it 5 out of 10. The scale is the strong part. 37,288 live servers is a real measurement, not a demo. It loses points because no journal has reviewed it, because the collection is not open to everyone, and because no outside group has picked it up yet.
What it does not mean
It does not mean the false alarm rate is 45.53 percent. It is the other way round. 45.53 percent is the share of alerts that were right. The false alarms are the rest, which is more than half. People flip these two constantly, and the flip makes the tools sound twice as good as they are.
It does not mean the figure is good to two decimal places. It rests on a hand check of 100 servers. Read it as “under half”. Do not carry the decimals into a slide, and do not treat a move from 45 to 48 as progress.
It does not mean your scanner is 45.53 percent right. That is an average of eight different tools. The spread runs from 10.40 percent to 96.88 percent. An average across tools that disagree is not a fact about any one of them. Find out which tool you actually run.
It does not mean the most accurate scanner is the one to buy. The scanner that was right 96.88 percent of the time hardly flagged anything. Precision only counts the alerts a tool raises. It says nothing about the problems it walked past. On a set of 10 published CVEs across 38 servers, the scanners caught 24.17 percent, and several caught none.
It does not mean 96.89 percent of MCP servers are dangerous. That flag rate is puffed up by copying. One template shows up 537 times in the collection. One weak design becomes hundreds of weak servers, so a frightening report can be the same server counted again and again.
And it does not mean MCP servers are fine. The paper does not say that. It says the current measurements cannot tell you. Meanwhile 37.6 percent of all the tools they found can run commands, change files or send data out. The power is real. The measuring is what is broken.
What it does mean
A scanner alert is a place to look. It is not evidence.
Here is why, from the paper. One scanner reported a leaked password because a field was named “token”. When the team ran that server, the field was an empty box waiting for input. No secret. No leak. Just a word that looked bad.
That is what most of these tools do. They read names, descriptions and shapes, then guess at danger. They rarely check whether an outsider could reach it at all.
You can see the guessing in how little they agree. Take any two scanners and their findings overlap 15.66 percent of the time. No server in the study was flagged by all eight. For prompt injection the overlap falls to 5.98 percent. For data leakage it is 3.36 percent. Eight tools look at the same server and mostly see eight different things.
Why it matters to you
You are the person who approves a new tool for your team. Someone hands you a scan report covered in red. You have two bad choices. Block everything, and the team routes around you. Wave everything through, and the alerts stop meaning anything.
A smoke alarm that screams at toast teaches you to ignore it. That is the risk this number describes. It is a bigger risk than any single bad server.
Two things follow. First, ask any scanner vendor two questions. How many of your alerts survive a human check? How many known bugs do you catch? Second, treat agreement as the signal. It is rare here. When two tools that look at different evidence both flag the same server, that one is worth your afternoon.
Do this today
Take the last MCP scan report you got. Pick one alert. Open the code at the line it names and answer one question. Could an outsider actually reach this? If you cannot answer in ten minutes, the alert was never specific enough to act on.
Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.
- 45.53 percent is the share of alerts that were real, so the share that were wrong is larger than half, and the two are often confused.
- The figure is an average of eight tools whose individual precision ran from 10.40 percent to 96.88 percent, so it is not a fact about the scanner you happen to run.
- The same eight scanners caught only 24.17 percent of 10 published CVEs across 38 servers, which shows precision alone cannot tell you whether a tool is worth having.
Questions people ask
Is 45.53 percent the false alarm rate?
No. It is precision, meaning the share of alerts that turned out to be real when people checked them. The false alarms are the remainder, which is more than half. Reading it the wrong way round makes the scanners sound about twice as reliable as the study found them to be.
Does this mean MCP servers are safe?
No. The paper says current measurements cannot tell you either way. It also reports that 37.6 percent of the tools it found can run commands, change files or send data out. The finding is about the quality of the warnings, not about the safety of the servers.
Which scanner should I use?
The study does not name a winner, and precision alone will not pick one. The most precise tool in the test hardly raised any alerts at all. Ask a vendor both how many of its alerts survive a human check and how many known bugs it catches, then treat agreement between two different tools as your strongest signal.
Can I look up a server in this study?
Partly. The paper links a public page where you can look up a server and see what the scanners said about it, along with how much they agreed. The full collection, called MCPZoo, is described as coming under controlled access for non-commercial academic use, so you cannot rerun the whole study yourself today.
Mark Alex
Mark Alex is the founder and Managing Director of Real Biz Digital, a technology company operating out of Nairobi since 2018. He works in agentic AI and the Model Context Protocol, AI governance, enterprise software architecture and cybersecurity. He holds an MSc in Mechatronical Engineering from Obuda University in Budapest and a BSc in IT, Forensic Technology and Cybercrime, from USIU-Africa in Nairobi, and has published IEEE conference research on an AI-powered digital twin for greenhouse systems. He is the author of seven books. Between 2020 and 2024 he mentored more than 200 university students and interns in Nairobi. He writes every Just Out Tech article from the original research paper.