A trial of 7,294 patients could only measure its own outcome in 17 percent of them

What happened

Up to half of older adults with cognitive impairment develop delirium after surgery. It is distressing, it lengthens hospital stays, and there is a known set of perioperative practices that reduce it. The problem is identifying which patients are at risk before the operation, when the anaesthesia team has minutes.

A team at the Icahn School of Medicine at Mount Sinai and the David Geffen School of Medicine at UCLA built a system to do it. Natural language processing scanned the electronic health record immediately before surgery to find signals of cognitive impairment, supplemented by Mini-Mental State Examination scores and diagnostic codes. Patients it flagged triggered a non-interruptive prompt in the record, nudging the anaesthesia team towards the twelve practices that help.

They randomised 7,294 patients. Delirium incidence did not change. Neither did adherence to any of the twelve practices.

And the trial ran into something more interesting than either result.

The test

Pragmatic randomised controlled trial embedded within routine clinical care at The Mount Sinai Hospital in New York City, from June 20, 2023 to August 31, 2024.

Patients with preoperative cognitive impairment were identified immediately before surgery and randomised to the AI-driven non-interruptive clinical decision support intervention within the electronic health record, or to routine perioperative care.

The primary outcome was postoperative delirium, assessed with the 4 A’s Test within seven days after surgery. The secondary outcome was adherence to twelve perioperative best practices.

The numbers

Measure	Intervention	Control
Patients randomised	3,624	3,670
Total randomised	7,294
With a documented delirium assessment	1,255 (17.2%)
Developed delirium, of those assessed	426 of 1,255 (34%)
Delirium incidence	225/641 (35.1%)	201/614 (32.7%)
P value, delirium incidence	.37
Adherence to 12 best practices	no significant difference, all P > .05

Look at the third row against the first. 7,294 patients entered the trial and 1,255 of them have a result for the thing the trial was built to measure.

The result

No statistically significant difference in postoperative delirium incidence: 35.1 percent intervention against 32.7 percent control, p = .37. No significant difference in best practice adherence, all p values above .05.

The authors’ own conclusion is unusually careful. Because less than 20 percent of the sample had documented delirium assessments, owing to variability in routine screening practices, they state that they cannot conclude the AI-driven decision support was ineffective at preventing delirium. What they can say, based on the secondary analysis, is that the intervention did not significantly change best practice adherence.

That second statement is the real finding. The nudge did not change what the anaesthesia teams did.

How this compares to what came before

Clinical decision support has a long history and a well-known failure mode: hard alerts that stop the clinician’s workflow get dismissed reflexively, and the dismissal rate climbs the more alerts you add. Alert fatigue is one of the most consistently replicated findings in health informatics.

The field’s answer has been non-interruptive design – information placed where the clinician will see it without being forced to act. It is now the default recommendation in most electronic health record guidance.

This trial is one of the few randomised tests of whether the gentler design changes behaviour at all. On twelve separate practices, across 7,294 patients, it did not. That is worth knowing, because the non-interruptive approach has been adopted widely on the strength of avoiding a known harm rather than demonstrating a benefit.

What it means

Two things, and the second is more important than the first.

The first is about nudges. A prompt that does not interrupt also does not compel, and in a perioperative workflow with a patient already on the table, it appears not to compete for attention. Hospitals deploying non-interruptive support should measure adherence, not assume it.

The second is about measurement infrastructure. This trial was well designed, adequately sized and embedded in real care, and it still could not answer its own primary question, because the bedside test that defines the outcome is not routinely done. You cannot run a trial of an intervention for a condition your hospital does not systematically detect.

That is a general constraint on clinical AI evaluation and it is under-discussed. A great many proposed AI interventions target conditions whose measurement in routine care is this patchy.

Business ideas from this paper

Outcome-readiness assessment. Before a hospital funds an AI trial, someone should check whether the outcome is documented often enough to analyse. That is a short, cheap piece of data work that would have saved this trial’s primary endpoint, and nobody sells it.

Delirium screening automation. The 4 A’s Test takes a couple of minutes and is skipped because wards are busy. A tablet-based prompt with structured capture, tied to a defined post-surgical window, would fix the measurement gap and is a far simpler product than the decision support itself.

Adherence instrumentation. This trial could measure the twelve best practices because they leave traces in the record. Building that measurement for other care bundles gives hospitals an outcome they can actually analyse, and it is the layer every subsequent trial will need.

Interruption design testing. Given that non-interruptive support did not move adherence here, there is room for a service that A/B tests alert design inside a hospital’s own record system rather than importing someone else’s conclusions.

What could be wrong with it

The primary outcome is not interpretable. Anyone citing this trial as evidence that AI does not prevent delirium is misreading it, and the authors say so themselves.

Even the patients who were assessed may be an unrepresentative subset – a ward that reliably performs the 4 A’s Test may differ in other ways that affect delirium.

One hospital, one city, one electronic health record configuration. Non-interruptive design is not one thing; it depends on where the prompt sits, what it says and what else is competing for the screen.

The secondary outcome is the firmer result, but twelve comparisons all returning p greater than .05 is consistent with a small effect that the trial could not detect as well as with no effect at all.

How sure can you be?

We score this 8 out of 10.

It is a peer-reviewed pragmatic randomised controlled trial in Anesthesia and Analgesia, embedded in routine care, with 7,294 patients randomised and named institutions at the Icahn School of Medicine at Mount Sinai and the David Geffen School of Medicine at UCLA.

It loses points because the trial ran at a single hospital with no independent site, and the data are not publicly deposited.

We would report the adherence finding with confidence and the delirium finding not at all. The authors’ willingness to publish a trial whose primary outcome did not work, and to explain exactly why, is the reason this scores as high as it does.

Do this today

If you are planning a clinical AI evaluation, run one query first: how often is your primary outcome actually recorded in your own data? Do that before you design anything else.

If your hospital has deployed non-interruptive decision support, pull the adherence numbers for the behaviours it was meant to change. Deploying it is not the same as it working, and this is the first randomised evidence that the distinction bites.

If you work in perioperative care, the delirium screening gap is actionable on its own, independent of any AI. A condition affecting up to half of a known high-risk group, assessed in 17 percent of them, is a problem you can fix without buying anything.

Source: Artificial Intelligence-Driven Clinical Decision Support for Postoperative Delirium Prevention: A Pragmatic Randomized Controlled Trial, Anesthesia and Analgesia, October 2026. PMID 42854247 · doi.org ().

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.