A veto worth having

Asked whether they would let software decide their diagnosis and treatment with no doctor reviewing it, fewer than one American patient in ten said yes. Add a single condition, that their own doctor could change or override the machine's decision, and the share rose to more than half. The survey, by West Health and Gallup and published on November 9, also asked primary care physicians to guess how their patients would answer. They guessed little more than a third.

Patients, in other words, fear the machine less than they fear the machine alone. The doctor's power to overrule is what makes medical AI acceptable to the public, and this week it was claimed, sold and stretched all at once. That veto is being spread thinner faster than anyone is paying to keep it real, and it should be protected in law, staffed in contracts and counted before it becomes a formality.

Doctors' organizations moved first to claim it. On November 9 the American Medical Association's House of Delegates, meeting in Orlando, voted that AI cannot hold a medical license, a physician's title or certification to practice, and that it belongs in teams led by physicians. The policy would put the cost of a defective product's errors on its vendor when a doctor has used it as intended, and on the hospital or insurer as well when they mandate a particular tool. It also seeks legal protection for physicians who decline to use an AI tool or to follow its advice. Dr. Willie Underwood III, the AMA's president, said that "patients deserve to know who is responsible for their care."

The veto is already exercised every day, with little preparation. In a Doximity survey of physicians in September, just over half said they used AI at work daily, and three in four had come across a factual error in its answers in the past year. Fewer than three in ten said their organization had clear policies and approvals for its use, and Gallup found that most primary care doctors had received no formal training in it at all. Doximity sells AI tools of its own, which makes its tally of errors the more striking.

Meanwhile the veto is being priced. On November 10 Oura, which makes a smart ring, agreed to pay $625 million in cash for Counsel Health, which lets patients move within minutes from its AI chat to a board-certified physician who can diagnose and prescribe. Counsel's chief executive describes its safety approach as "keeping physicians in the loop for clinical decisions." The announcement cited satisfaction scores, no clinical outcomes and no figure for how many patients each of its doctors looks after.

Inside hospitals the same arithmetic is at work. HCA Healthcare expects about 10,000 of its physicians to be using an ambient scribe, software that listens to a visit and drafts the note, by the end of the year, and its chief clinical officer, Dr. Michael Schlosser, told Becker's that it saves one to two hours a shift. Becker's reported that outpatient clinics have seen more throughput as physicians chose to add patients to their schedules. Google opened a preview of an agent that reads records in Epic and billing systems and prepares prior authorizations, the paperwork insurers demand before they will pay for a treatment; it can read records but not change them, leaving decisions to a doctor or nurse. Northwell Health's chief executive, John D'Angelo, told Fortune that he has about 400 employees whose sole job is fighting insurance denials. An analysis of Medicare billing in NEJM AI found that family and internal medicine have among the least physical protection from substitution by AI, so the hours a machine frees in those fields are the easiest to refill.

The week's evidence suggests the human check still earns its keep. Across more than two million encounters at 15 Danish hospitals, a machine-learning warning score added almost nothing to the National Early Warning Score, a simple tally of vital signs, except in the sickest patients. Dr. Roy Perlis of Massachusetts General Hospital showed that a hidden adjustment to a chatbot's internal workings could steer its depression advice away from antidepressants, a change he wrote would be invisible to a patient or a clinician. A veto cannot catch what it cannot see, which is why the AMA also wants vendors to disclose their products' limitations and performance data.

Technologists will reply that a veto exercised on every decision defeats the purpose. If software is right more often than a tired doctor, a mandatory human check preserves the bottleneck and the cost that AI was meant to remove, and patients' comfort with autonomy will grow as the tools prove themselves. That may well happen. But comfort should follow evidence, and the week supplied little of it: Counsel's announcement reported no outcomes, the Danish model barely improved on a simple vital-sign score outside the sickest patients, and a randomized trial of a maternal-health chatbot found no effect on anxiety or depression at a year. Until products show results in ordinary practice, the doctor's override is the main safeguard patients have, and an override squeezed between extra appointments protects nobody.

Legislators should enact what the AMA asked for: no liability for a physician who reasonably declines an AI recommendation, and no contract that shifts a defective product's failures onto the doctor who used it. Insurers should adopt the rule Alabama has written into law, that a licensed physician or other health professional makes every prior-authorization denial on grounds of medical necessity. Hospitals and medical staffs should write the clear AI policies that most physicians in the Doximity survey did not report having, train clinicians before a tool is switched on, and decide in writing, before an ambient scribe goes live, how much of the time it saves goes back into the schedule. Companies that sell care backed by a physician should publish how many patients each doctor oversees and how often doctors overrule the machine, and physicians should ask their employers for the same two numbers this month.

Patients have named their price for accepting the machine: a doctor who can say no. Medicine should make sure that doctor has the time, the training and the legal cover to say it.