The thinking is the job
A new paper says medical training should teach doctors when to defer to a machine. It has the order backwards, and almost nobody is counting what the deferring is worth.
Listen to this special topic
read by an AI voiceFour emergency physicians at the University of Texas Southwestern Medical Center published a perspective on Oct. 13 with a line that should stop any doctor cold. One day, they write, the most dangerous actor in the room may not be the machine making a recommendation. It may be “the human who insists on overruling it simply because being in the loop still feels like being in charge.”
They may be right about that doctor. The trouble is the paper they wrote to get there. It asks medical training to stop prizing struggle for its own sake and refocus it from recalling facts toward “interrogating machine reasoning, detecting failure modes” and knowing when a person should step in.
Deference is not a skill of its own. It sits on top of one. The same paper says so. A trainee who has never had to defend a differential diagnosis “may be poorly prepared to recognize when an AI system is incomplete, overconfident, or wrong”. You cannot teach a young doctor to tell a good recommendation from a confident wrong one until she can tell those apart on her own. The struggle is not a teaching method we picked and could swap for a better one. It is the thing being taught.
Which raises the question almost nobody is answering. If the doctor in the loop is the safeguard, how much does it catch? A systematic review of these systems in primary care, published this month, looked at eleven studies. Not one reported whether the clinician agreed with the machine or overrode it. Three of the perspective's six references are commentary, and the five that can be read follow no patient past the moment a doctor accepted or refused a machine's advice.
The one randomized trial among them ran at the end of 2023. 50 physicians, nearly half of them residents, got an hour and up to six written cases, with ChatGPT or their usual references. Blinded scorers graded the reasoning they wrote down. With the model, a median of 76 percent per case. Without it, 74. Two points, and the trial could not separate that from chance. The model working the same cases alone scored 16 points higher. Now read what that trial is not. Nobody took a history. Nobody examined anyone. Nobody found out what happened to a single patient. The model's score came from a prompt a researcher wrote and ran three times, in what the authors call a secondary exploratory analysis. They say the rubric score is a signal and that “broader clinical trials are necessary to assess for meaningful differences in downstream clinical impact.”
Deskilling and automation bias are visible wherever someone bothers to measure them. 27 radiologists read 50 mammograms with ratings attached that they were told came from AI. When the suggestion was wrong, the least experienced readers went from getting about four in five right to one in five. The most experienced dropped below half. In four Polish endoscopy centres, adenoma detection in colonoscopies done without the software fell from 28.4 percent to 22.4 percent in the three months after it arrived. Its authors call that a possible effect of continuous exposure. Not forgetting how to find a polyp. Not looking as hard. One arrangement has held up in real care. A Swedish trial screened nearly 106,000 women. Software sorted the mammograms and flagged findings. Radiologists still decided whom to call back. Detection rose from five cancers per thousand to six, with no more false positives, reading work almost halved and cancers surfacing between screens no worse. Public money and the Swedish Cancer Society paid for it. What was tested there was the arrangement. Not the override.
Rachael Bedard, a geriatrician who uses OpenEvidence every day, wrote in a New York Times guest essay on Sept. 25 that it has made her a better doctor and may be making her trainees worse, and argued for teaching them to think critically with a machine in the loop. Adam Rodman, quoted in the same piece, put the line where it belongs. A little deskilling in experienced hands probably does not matter much. In someone with far less skill, “the cognitive offloading can cause never-skilling.” One of those is a bad habit. The other is a doctor who never arrives.
The blunt case for getting out of the way is not Bedard's. In JAMA in August, Ezekiel Emanuel, Abe Baker-Butler and Neal and Vinod Khosla of Curai Health and Khosla Ventures asked whether autonomous AI will beat AI-aided physicians. Where that turns out to be true, a doctor standing in the way costs patients something real, and the perspective is right that a profession should not dress up its own indispensability as a moral principle. But look at the job it offers instead. Orchestration, it says. Among other things, routing requests, reconciling contradictory outputs, watching for deterioration. That is the most automatable work in the building.
So count it. A hospital that puts a doctor between a model and a patient should record what the doctor did and what followed, and how those patients fared against the rest. A monthly chart sample is within reach of a small hospital. Short of a trial, it is the only way to learn whether the reviewer is a safeguard or a signature. Put a plausible, wrong recommendation in front of a trainee and grade the catch, as the perspective itself asks. The Association of American Medical Colleges has AI competencies targeted for this fall, and that belongs in them. Funders should pay for the prospective trials these authors keep asking for.
Teach the foundations first. They are the only thing that separates a doctor from a layman with a search box, and a profession that hands them over will not be asked to stay. The perspective may be right about the doctor who overrules a better system out of habit. Nobody can say how often, because almost nobody is counting. We are not defending mastery. We are not yielding wisely either. We are guessing, and telling students it is a curriculum.
- The perspectivePhilip Jarrett, D. Mark Courtney, Deborah Diercks and Samia Farooqi, all of the Department of Emergency Medicine at the University of Texas Southwestern Medical Center in Dallas, published "No Mastery without the Machine", a perspective on medical education in the age of clinical AI, in NEJM AI 2026;3(11), DOI 10.1056/AIp2600773. It was received June 6, 2026, revised Aug. 26, accepted Sept. 9 and published Oct. 13, 2026. It is a perspective article rather than a study, and it carries six references. Its abstract says medical mastery may shift toward "the orchestration, supervision, and calibrated deference to intelligent systems." The authors state that ChatGPT Images 2.5 (OpenAI) was used to assist with image generation and that no AI tools were used for writing assistance; author disclosures are posted at ai.nejm.org.Source: NEJM AI
- The randomized trial of a language model and physician reasoningGoh and colleagues ran a single-blind randomized trial from Nov. 29 to Dec. 29, 2023 with 50 licensed physicians (26 attendings, 24 residents; median 3 years in practice), recruited at Stanford, Beth Israel Deaconess and the University of Virginia. In one-hour sessions they worked up to six written vignettes based on actual patients, randomized to ChatGPT Plus (GPT-4) with conventional resources or to conventional resources only; 244 cases were completed. The primary outcome was a structured-reflection rubric graded by blinded scorers. The median score per case was 76% with the model (IQR 66 to 87) and 74% without (63 to 84), an adjusted difference of 2 percentage points (95% CI minus 4 to 8; P=.60). The model alone, prompted by a researcher physician and run three times, scored a median 92% and 16 percentage points above the control group (95% CI 2 to 30; P=.03). Residents were offered $100 and attendings up to $200 to take part. PubMed indexes the trial under N.I.H. extramural research support and research support from outside the U.S. government; Eric Horvitz of Microsoft is among its coauthors. JAMA Network Open 2024;7(10):e2440969.Sources: JAMA Network Open, PubMed record
- What that trial's authors say it cannot showThe paper states that the cases "have never been publicly released to protect the validity of the test materials for future use, and therefore are excluded from training data of the LLM." Its discussion says the results should not be interpreted to indicate that language models should be used for diagnosis autonomously without physician oversight; that the curated vignettes do not capture competence in patient interviewing and data collection; that the study was acontextual, while clinicians' understanding of the clinical environment is fundamental to high-quality decision-making; and that "broader clinical trials are necessary to assess for meaningful differences in downstream clinical impact." Among the limitations, participants were given the chatbot without explicit training in prompt engineering, which the authors say is consistent with current integrations.Source: JAMA Network Open
- The capability study in ScienceBrodeur, Buckley, Kanjee and colleagues, with Adam Rodman as senior author and Eric Horvitz of Microsoft among the coauthors, published "Performance of a large language model on the reasoning tasks of a physician" in Science 2026;392(6797):524-527 on April 30, 2026, DOI 10.1126/science.adz4433. The abstract describes a physician evaluation of a model on challenging clinical cases across five experiments with a baseline of hundreds of physicians, and a real-world study comparing human expert and AI second opinions in randomly selected patients in the emergency room of one major tertiary academic medical center. It reports that the model outperformed physician baselines in all experiments, and that the results suggest models have eclipsed most benchmarks of clinical reasoning, "motivating the urgent need for prospective trials."Sources: Science, PubMed record
- The education papers on either sideKe, Jin, Ong and 13 coauthors led from Duke-NUS Medical School published "AI-induced never-skilling in medical education" in Nature Medicine 2026;32(6):1997-2006 on May 22, 2026, DOI 10.1038/s41591-026-04438-y. It separates never-skilling in trainees from deskilling in experienced clinicians and from mis-skilling, states that "direct evidence from medical training is absent" while the concern rests on established learning theory and early signals from nonclinical settings, and proposes a three-phase framework that begins with AI-independent baseline competency. The JAMA article the perspective answers is Ron Keren, Bimal Desai and Daniel West, "Promoting Clinical Expertise in the Age of AI: No Struggle, No Mastery," JAMA 2026;335(21):1837-1838, June 2, 2026. The Association of American Medical Colleges says on its own page that AI competencies for medical education are in development, with a final report targeted for fall 2026; the page does not set out how they would be assessed.
- The screening trial that tested an arrangementIn the Swedish MASAI trial, 105,934 women were randomized 1:1 between April 12, 2021 and Dec. 7, 2022 at four sites in southwest Sweden; software (Transpara 1.7.0, ScreenPoint Medical) triaged examinations to single or double reading and highlighted findings, and radiologists decided recall. Detection, recall, false positives and reading workload are a protocol-defined analysis of secondary outcome measures: cancer detection was 6.4 per 1,000 screened (95% CI 5.7 to 7.1) with AI support and 5.0 (4.4 to 5.6) with standard double reading, a ratio of 1.29 (1.09 to 1.51; P=.0021); recall and false-positive rates were not significantly higher; screen readings fell from 109,692 to 61,248, a 44.2% reduction in reading workload. The protocol-defined primary outcome, interval cancer, carried a 20% non-inferiority margin and was 1.55 versus 1.76 per 1,000 (ratio 0.88, 0.65 to 1.18; P=.41), which it met, with sensitivity 80.5% versus 73.8% (P=.031) and specificity 98.5% in both groups. Funding: Swedish Cancer Society, Confederation of Regional Cancer Centres and Swedish governmental funding for clinical research.
- Deskilling measured in patients, automation bias measured on imagesBudzyn, Romanczyk and colleagues report a retrospective observational study at four Polish endoscopy centers taking part in the ACCEPT trial, which introduced AI polyp detection at the end of 2021. Among 1,443 diagnostic colonoscopies done without AI, 795 in the three months before and 648 in the three months after, the adenoma detection rate fell from 28.4% (226 of 795) to 22.4% (145 of 648), a difference of minus 6.0 percentage points (95% CI minus 10.5 to minus 1.6; P=0.0089); exposure to AI carried an odds ratio of 0.69 (0.53 to 0.89). The authors say continuous exposure might reduce the rate, which keeps the finding tentative. Funding: European Commission and the Japan Society for the Promotion of Science. Lancet Gastroenterology and Hepatology 2025;10(10):896-903. Separately, in a prospective experiment in Radiology 2023;307(4):e222176, 27 radiologists read 50 mammograms with categories suggested by a purported AI system; where the suggestion was incorrect, mean correct ratings fell from 79.7% to 19.8% among inexperienced readers, 81.3% to 24.8% among moderately experienced readers and 82.3% to 45.5% among very experienced readers.
- The argument for autonomy, and the surgical citationEzekiel Emanuel, Abe Baker-Butler, Neal Khosla and Vinod Khosla published "Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care?" in JAMA 2026;336(11):915-918 online on Aug. 17, 2026 and in the issue of Sept. 15, DOI 10.1001/jama.2026.15380. JAMA lists Emanuel and Baker-Butler at the University of Pennsylvania, Neal Khosla at Curai Health and Vinod Khosla at Khosla Ventures; PubMed's record for it carries no abstract. MDLinx reported on Sept. 25, 2026 that an essay by Emanuel and Vinod Khosla in The Atlantic drew replies from clinicians, among them the chief executive of the American Medical Association. The perspective's surgical citation is Kim and colleagues, Science Robotics 2025;10(104):eadt5254, which reports a 100% success rate across eight ex vivo gallbladders, that is, in tissue outside the body rather than in patients.
- The essay from the other directionRachael Bedard, "Being a Doctor Will Never Be the Same After A.I.", a New York Times Opinion guest essay, Sept. 25, 2026; the print version ran Sept. 27 in Section SR, page 12, headlined "A.I. Has Made Me a Better Physician". The Times identifies her as a contributing Opinion writer, a geriatrician and palliative care physician. She describes using OpenEvidence routinely, says her students and residents use it too, rejects shielding medicine from the tool, and argues for training physicians to think critically with a machine in the loop while keeping the instincts she calls cardinal: telling sick from not sick and drawing out what a patient does not volunteer. She reports proposals from Adam Rodman for preserving friction, among them AI-free clinical reasoning sessions and AI literacy training, and repeats the company's own claim that over half of American doctors use the app regularly. That tool is the subject of this publication's Oct. 28 piece, "Nobody has graded the answers". The essay was read by the publisher in his own browser; this publication's own tools could not open the page.Source: The New York Times