Shadow AI Is Already in the Hospital — and No Risk Register Knows Its Name

Somewhere this morning, a doctor pasted a half-formed discharge summary into a chatbot on a personal phone, read the tidy version it produced, edited a line, and copied it back into the record. No procurement process touched that exchange. No clinical safety case covered it. No governance committee knows it happened. By the time the relevant committee meets to decide whether the organisation is ready to adopt AI, the question will already be several months out of date — because the staff adopted it without asking.

This is shadow AI, and the phrase flatters the novelty of the thing. It is not a coming risk to be headed off with a policy. It is a present fact, distributed across thousands of devices that the institution does not control and cannot see. The committees are deliberating about a future that has already arrived through the side door. And the gap between the official position — we are evaluating our options — and the operational reality — people are using it daily — is exactly the kind of gap where harm accumulates quietly, because nobody with authority is looking at it.

What the shadow actually looks like

Strip away the alarm and look at the usage on its own terms, because it is more mundane and more reasonable than the word "shadow" suggests.

A registrar facing a backlog of letters drafts a referral by describing the case in plain language and asking for it back in the right register. A clinician who has not seen a particular rare condition since training types its name into a chatbot for a fast orientation before reading anything more authoritative. A doctor rewrites discharge instructions at a reading level a worried relative can actually follow, or renders them into a language the patient speaks and the clinician does not. Someone turns four lines of clipped clinical shorthand into a paragraph a coroner would find legible. Someone else asks for the differential they might be anchoring away from, as a check on their own thinking.

None of these are exotic. Every one of them is a real task that the job demands and that the working day does not allow enough time for. The tools are good at exactly this register — fluent prose, fast reformatting, plausible structure — and they are free, and they are already on the phone that is already in the pocket. The usage is not a transgression dreamt up by the reckless. It is competent people reaching for the nearest thing that works, which is what competent people under load have always done.

That is precisely why it is so hard to stop, and why pretending it isn't happening is the least useful available posture.

Why it happens, and why a memo will not end it

Two forces meet here, and a policy memo opposes neither of them.

The first is that the tools crossed the threshold of being genuinely useful while remaining trivially accessible. For most of computing history, a capability that good required procurement, training, a login, a budget line. This one requires a thumb. When the marginal cost of a useful tool drops to nothing and the friction drops to a tap, usage does not wait for permission. It diffuses. The economics of how technology spreads have changed underneath the institutions that still assume adoption is something they grant.

The second is that the sanctioned alternative is, in most places, either absent or worse. There is frequently no approved tool that does the same job, and where one exists it is often slower, clumsier, gated behind a login that times out, or missing the one feature the shadow tool nails. A clinician at hour nine of a shift is not running a procurement evaluation. They are choosing between the thing that drafts the letter in twenty seconds and the thing that does not exist. Utility outruns policy — as it has before, when personal cameras quietly became the ward's clinical photography long before anyone sanctioned it, for the same unglamorous reason: the official channel was not there when the work was.

A memo declaring the shadow tools forbidden changes none of this. The task is still there. The deadline is still there. The tool is still in the pocket. All the memo changes is whether the usage is discussed.

The risks, stated plainly — and what the ban reflex does to them

This is the part that the honest version of this argument cannot skip, because the risks are real and specific, and minimising them would be its own kind of dishonesty.

Confidentiality is the sharp one. A consumer chatbot is not a clinical system. Identifiable patient information typed into it leaves the controlled environment and enters a third party's infrastructure under terms the clinician has not read and the organisation has not negotiated. This is not a hypothetical about model training; it is a present fact about where the data goes. It is the most concrete hazard in the entire picture, and it deserves to be named without hedging.

Accuracy is the quieter one. These systems produce fluent, confident text whether or not the content is sound, and they confabulate in a register indistinguishable from the register they use when correct. An unverified output pasted into a record does not arrive flagged as unverified. It arrives looking exactly like everything around it — and the fluency is precisely what makes a tired reviewer likely to wave it through.

Invisibility is the one nobody feels until later. Because the usage is off the books, there is no audit trail, no error surveillance, no way to learn from the near-miss, no signal when a particular failure mode starts to recur. The institution cannot study a practice it has officially decided is not occurring. Every other safety-critical process in medicine improves by being watched; this one is structurally exempt, not because it is safe, but because it is hidden.

And here is the trap. The instinct, on reading that list, is to ban. But the ban does not remove the confidentiality risk — it removes the conversation about the confidentiality risk. The clinician still needs the letter drafted, still reaches for the tool, and now does so without ever raising a hand to ask which corners are safe to cut. Prohibition does not reduce the usage. It reduces the visibility of the usage, which means it makes every one of the three risks worse while producing a document that lets the organisation feel it has acted. The ban is the risk estimate masquerading as the control: a hazard with a sticker on it, and the sticker reads not permitted.

Governing the thing that already exists

The alternative is harder than a prohibition and considerably more useful than one. It starts from a premise the committees resist: you cannot govern what you refuse to admit is happening.

Begin by measuring honestly. An anonymous survey that asks clinicians what they actually do — without a disciplinary edge to it — will return a picture sharply at odds with the official one, and that picture is the only sane starting point for any policy. Governance built on the fiction of zero usage governs nothing. You cannot manage what you have decided not to count.

Then draw red lines that are few, bright, and genuinely defensible. No identifiable patient data into consumer tools is a line clinicians can understand, remember, and keep, because it maps onto a duty they already hold. A line that broad and clear does more real protective work than a forty-page acceptable-use policy nobody finishes reading.

Then — and this is the part institutions keep skipping — provide a green lane that is actually as convenient as the shadow one. A sanctioned tool that is slower, uglier, or gated behind friction will lose to the thing in the pocket every time, and deservedly so, because the clinician's problem is real and the official answer has to solve it at least as well. The convenience of the shadow tool is not a moral failing to be lectured at. It is a product specification for the thing the organisation ought to supply. Matching it is the whole game.

What this produces is not the elimination of shadow AI, which is not on the menu. It is the conversion of invisible use into governed use — the same usage, now inside a structure that can see it, set limits on it, learn from it, and improve.

What this means

The choice was never between AI in the hospital and no AI in the hospital. That decision was made, in aggregate, by the staff, months ago, one pasted paragraph at a time, and no committee was consulted. The only choice that remains is between use the institution can see and use it cannot — between drawing a usable line around a real practice and pretending the practice into a darkness where it goes on exactly as before, now with no light on it at all.

Pretending is the worst option, and it is the one most organisations are currently choosing by default. The shadow does not grow because clinicians are reckless. It grows because the work is hard, the tools are good, the official answer is missing, and the institution would rather deliberate about a hypothetical adoption than govern the real one already underway. The hospital does not get to decide whether shadow AI exists. It only gets to decide whether it will keep its eyes open.