AI in healthcare needs more than a model — it needs a record

Healthcare is where the gap between an impressive model and a trustworthy system is widest. A model that reads a scan or suggests a diagnosis can be genuinely brilliant and still be something you'd never let act on its own — because the cost of a confident mistake isn't an awkward email. It's a person.

So the interesting work in medical AI is rarely the model itself. It's everything around it: who is accountable for the recommendation, how a clinician stays in control, what happens to the patient's data, and whether anyone can reconstruct, months later, exactly what the system did and why.

The rules are about trust, not paperwork

In Ontario, PHIPA governs how health information can be collected, used, and disclosed, and it doesn't loosen because the thing touching the data is a model. When AI is built into a medical device, Health Canada regulates it under the Food and Drugs Act, and the international Good Machine Learning Practice principles — published jointly by Health Canada, the U.S. FDA, and the U.K.'s MHRA — put human oversight and traceability near the centre. The common thread is the one that runs through all of this: capability is necessary, but it's accountability that earns trust.

Keep the clinician in the loop — really in it

"Human in the loop" is said so often in health AI that it's worth being precise about what it has to mean here. Not a clinician rubber-stamping outputs they have no time to question. A clinician who can see what the system is proposing, has the context to judge it, and has both the authority and the time to override it. A reviewer who can't understand the recommendation, or can't say no without being penalised for slowing things down, is decoration — and in medicine, decoration is dangerous.

• Consent and purpose — patient data used only for what it was gathered for, with PHIPA's limits enforced, not assumed.
• Human authority — a clinician can override any recommendation, and the system makes that easy rather than awkward.
• Traceability — every suggestion and action recorded immutably, so an adverse event can actually be investigated.
• Proportion — oversight scales with the stakes; a scheduling hint and a treatment suggestion are not governed the same way.

Where Eloryn fits

Eloryn's demo includes a health authority among the example agents it governs, held to health-data rules rather than generic ones. The platform — built and deployed by iiSP — adds restraint and proof: a capability sandbox so a model can only touch what it's been granted, a human-oversight step on anything high-stakes, and a signed, tamper-evident record, so that when someone asks what happened to a particular patient's file, there's a real, inspectable answer.

In medicine the question isn't whether the AI is smart. It's whether, when something goes wrong, you can find out exactly what it did.

That's the bar clinical AI has to clear before it earns a place near patients. Not perfection — nothing in medicine is perfect — but accountability you can actually exercise. Build that in, and AI can take a real load off an overstretched system without anyone having to take it on faith.