Clinicians moved 11 percent of patients into the beds with the AI screen, and broke the trial

What happened

Predictive deterioration monitoring is one of the most widely installed categories of hospital AI. The system watches vital signs and other data, computes a risk score, and shows it. The theory is that a nurse or doctor sees a rising line and intervenes before the patient crashes.

A team at the Center for Advanced Medical Analytics at the University of Virginia put that theory through a pragmatic randomised controlled trial on an 85-bed inpatient cardiology and cardiac surgery ward. 10,422 inpatient visits were randomly assigned by cluster: some beds showed a display of risk trajectories, some did not. There was a substantial implementation and education plan behind it.

The primary outcome did not change. And something else happened that is arguably more useful than the headline result.

The test

The display was passive. It showed risk trajectories with no specific response mandated. Nobody was required to do anything when a line went up.

The primary analysis compared events of clinical deterioration – death, emergent ICU transfer, emergent endotracheal intubation, cardiac arrest, or emergent surgery – and 21-day mortality after admission. The outcome of interest was hours free of clinical deterioration events.

Randomisation was by cluster at the bed level, which is how a ward-based trial of a bedside display is usually done. It is cheap and it does not require consenting individual patients.

The numbers

Measure	Finding
Inpatient visits randomised	10,422
Ward	85-bed cardiology and cardiac surgery
Primary outcome (deterioration events)	no change between groups
Event-free hours among those with an event	more in the display-on group, not statistically significant
Length of stay, large risk-score spike	6.8 days
Length of stay, no large spike	3.4 days
Patients transferred into or out of display beds	11%
Registration	NCT04359641

The result

No change in the primary outcome. Among patients who did have a clinical event there were more event-free hours in the display-on group, but the difference did not reach statistical significance.

The model itself discriminates: patients with a large spike in risk score had, on average, twice the length of hospital stay, 6.8 days against 3.4.

Then the finding that makes this paper worth reading. Clinicians chose to transfer 11 percent of patients into or out of display beds. Each transfer was a censoring event, removing that patient from the analysis, and the authors state plainly that this undermined aspects of the randomised nature of the study. The pragmatic design allowed significant movement of sicker patients towards the beds with displays.

The authors conclude that predictive analytics monitoring with displays of risk trajectories, coupled with an education plan, did not improve patient outcomes, and that future design considerations must focus on understanding clinicians’ interpretation, care processes and communication practices.

How this compares to what came before

The evidence base for deterioration prediction is overwhelmingly retrospective and overwhelmingly about discrimination: area under the curve, sensitivity at a chosen threshold, how early the score would have flagged patients who went on to crash. On those measures these systems look excellent, and this one does too.

Randomised trials of whether showing the score changes outcomes are rare, and the few that exist have been mixed. This one adds a specific and transferable observation: in a bed-randomised design, clinicians will reorganise the ward around the information, and that behaviour will corrupt the comparison.

That is not a flaw in the trial so much as a finding about how clinicians respond to being given a tool in half the ward. It will recur in any similar design.

What it means

For hospitals running deterioration prediction, the practical question is what happens when a score rises. If the answer is “it appears on a screen”, this trial suggests you should not expect an outcome benefit. Information without a protocol is not an intervention.

For the people evaluating these systems, bed-level randomisation is now known to be fragile. The transfers were not misconduct; they were clinicians doing their jobs with the tools in front of them. Any trial design that lets the staff route patients towards the intervention will produce this problem.

And the discrimination result is a reminder that a good model and a good system are different things. Twice the length of stay for patients the model flags means the score carries real signal. The signal went nowhere.

Business ideas from this paper

Response protocols as the product. The missing layer is not a better score; it is a defined, auditable action when the score crosses a threshold – who is called, within how long, what they do. Selling the protocol and its audit trail alongside the model is the gap this trial exposes.

Escalation-pathway auditing. Hospitals that already own these systems can be shown how often a high score was followed by any documented clinical action. Most will not know. That audit is a short engagement with an uncomfortable and valuable answer.

Trial design services for ward-level technology. The bed-randomisation trap in this paper will catch the next team too. There is a niche in designing and running evaluations that survive clinician behaviour, and hospitals cannot build that expertise internally for one study.

Transfer-aware analytics. The 11 percent movement rate is itself a signal about which patients clinicians are worried about, independent of the model. Instrumenting it would give a ward a second, human-generated risk indicator that nobody currently captures.

What could be wrong with it

The randomisation was compromised and the authors say so. Eleven percent of patients moved between arms, and they were not a random eleven percent – sicker patients were moved towards the displays. That biases the comparison in the direction of making the display look worse.

The display was passive by design. This trial tests one weak implementation of a system class, not the class. A version that pages a rapid response team at a threshold would be a different intervention entirely.

One 85-bed ward in one academic hospital, in cardiology and cardiac surgery. Deterioration patterns and staffing differ sharply across specialties.

The secondary signal – more event-free hours in the display-on group among those who had an event – was in the direction of benefit and did not reach significance. With a corrupted randomisation, it is hard to know what to do with that.

The data are not publicly available; the authors state they are still analysing secondary outcomes and will share for replication on request.

How sure can you be?

We score this 7 out of 10.

It is a peer-reviewed pragmatic randomised controlled trial in Scientific Reports, covering 10,422 inpatient visits, registered on ClinicalTrials.gov, from a named institution – the Center for Advanced Medical Analytics at the University of Virginia School of Medicine.

It loses points because it ran at one institution on one ward with no independent site, the data are not openly deposited, and the randomisation was partly undone by patient transfers.

That last point cuts both ways. It weakens the inference and it is the most useful thing in the paper, because the authors documented it rather than quietly dropping the censored cases.

Do this today

If your hospital runs a deterioration score, find out what is supposed to happen when it rises, and then find out what actually happens. The gap between those two is where this trial’s null result lives.

If you are designing a ward-level evaluation of anything, assume clinicians will move patients towards the thing they believe helps. Randomise patients if you possibly can, and if you cannot, plan for how you will handle crossovers before they happen rather than after.

If you are buying predictive monitoring, ask the vendor for randomised outcome evidence, not discrimination statistics. The discrimination in this trial was fine. The outcomes did not move.

Source: A randomized controlled trial of artificial intelligence-based analytics for clinical deterioration, Scientific Reports, February 2026. PMID 41644641 · doi.org ().

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.