Wait for the Evidence
Calling it malpractice not to consult AI assumes doing so improves diagnoses, treatment and outcomes. Outside a few tasks the evidence does not yet show that; the standard of care should follow the trials.
Listen to this special topic
read by an AI voiceAt a summit in Washington on Sept. 29, the health secretary, Robert F. Kennedy Jr., said artificial intelligence can give a patient a second opinion "much better informed than any doctor in the country." He told of a visit from OpenAI's chief executive, Sam Altman, who, by Mr. Kennedy's account, had said that "it would be malpractice for a doctor to make a diagnosis or to make a prescription without at least checking AI." Earlier at the summit an OpenAI executive, Felipe Millon, told the audience, "It is medical malpractice not to get a second opinion from AI today," Medical Economics reported.
Mr. Kennedy's reasons were time and volume: a patient's record can run to a thousand pages, he said, and a doctor with six minutes for a visit cannot review it, while software can. That is an argument about reading. A duty to consult AI before every diagnosis and prescription rests on a larger claim: that doing so makes diagnoses more accurate, treatment better and patients better off. The transcript of his remarks cites no study for it, and the report of Mr. Millon's remark does not show him citing one. Outside a few specific tasks, the evidence does not yet support it. The standard of care has moved toward what a competent physician would do in light of available knowledge and resources, and until trials show, task by task, that patients do better when their doctors consult AI, not consulting it is no lapse from that standard.
Each part of that claim can be tested, and the kind of test matters. Most of what is known about language models, the software behind chatbots such as ChatGPT, comes from exams, benchmarks and written cases. Of the studies in clinical medicine published from 2022 to September 2025, a review in Nature Medicine found that most tested the models on simulated cases or exam-style questions; of the roughly 1,000 that used real patients' data, only 19 were prospective randomized trials. Tests on paper show what a model knows, not whether patients do better when doctors use it.
On diagnosis, the best models now beat doctors on paper, often under rules that handicap the doctors. In a randomized, blinded comparison published in Nature last year, Google's diagnostic chatbot, AMIE, made more accurate diagnoses than primary care physicians when each held text consultations with actors playing patients, a format the authors call unfamiliar in clinical practice. In a preprint, Microsoft reported that its own diagnostic system solved more than four in five hard cases from the New England Journal of Medicine, against one in five for generalist physicians working without colleagues, textbooks or other AI. And in a randomized trial on written cases, run by researchers at Stanford, Beth Israel Deaconess and elsewhere, an exploratory analysis found that GPT-4 working alone outscored physicians using their usual resources by 16 points.
Giving a model to doctors has helped less. In that trial, physicians given GPT-4 reasoned no better about diagnosis than those with their usual resources, and in a randomized study of hospital clinicians reading case vignettes, a deliberately biased chest X-ray model cut their diagnostic accuracy by 11 points. Real clinics have been studied far less. In a quality improvement study with financial support from OpenAI, Penda Health, which runs clinics in Kenya, gave an AI assistant to a randomly chosen half of its clinicians, and reviewing physicians judged that those with the assistant made fewer diagnostic errors. In a randomized trial in Penda's clinics, published in June, clinicians with the assistant recorded an appropriate diagnosis more often. Google's AMIE has been tried with 98 patients at Beth Israel Deaconess, in a study published in The Lancet this month; its authors said it was designed to assess feasibility, not whether AI improves health outcomes.
On treatment, the evidence on paper is somewhat more favorable. OpenAI says the version of ChatGPT it built for clinicians outperformed physicians who had unlimited time and web access, on its own benchmark, drawn from doctors' chats about patient care, documentation and research. The benchmark's authors note that it was weighted toward cases OpenAI's models find hard, so its absolute scores are not directly comparable to real-world performance rates. In a second trial by the Stanford and Beth Israel group, physicians given GPT-4 scored six and a half points higher on written cases about managing patients' care. In Penda's clinics, reviewing physicians judged that clinicians with the assistant made fewer treatment errors, yet in the randomized trial, treatment failure within two weeks, its main measure, was about as common with the assistant as without it, in roughly one case in fifty.
On outcomes, which matter most, the assistant in Kenya has yet to show a gain; in the earlier study, too, patients' own reports of whether they felt better did not differ significantly. Narrower tools have done better. In one randomized trial in Taiwan, software read hospital patients' electrocardiograms and alerted their doctors to those at high risk; deaths within 90 days fell by about a sixth, though its authors cannot yet fully explain why. In Sweden, a randomized trial of breast screening found that mammograms read with AI support caught more cancers and cut radiologists' reading by more than two-fifths, while no more cancers surfaced between screenings than under the usual reading by two radiologists.
Even where AI plainly finds more, the gain for patients can stay unproven. In randomized trials pooled for the gastroenterologists' guideline, software that flags polyps during colonoscopy found precancerous growths in about a fifth more patients, mostly tiny ones with little potential to become cancer. Yet the American Gastroenterological Association made no recommendation for or against it last year, saying its impact on preventing colorectal cancer "remains unclear." The companies' own accounts concede the gap. Microsoft's preprint says its results "do not yet establish the clinical efficacy" of its system, and Google's account of the clinic study says "larger clinical trials are needed to assess patient-facing AI at scale." OpenAI wrote last year of the assistant in Kenya: "Follow-up studies are needed to further study how these copilots affect patient outcomes."
Waiting has its own toll, and that is the best case for expecting doctors to consult AI now. By one estimate, diagnostic errors kill or permanently disable nearly 800,000 Americans a year. The models already beat doctors on hard cases, the argument runs, and every year spent on trials means preventable harm. The toll is real, and a reason to run trials quickly rather than to skip them. Beating doctors on paper is not the same as reducing that toll: in Kenya, better-recorded diagnoses did not bring fewer treatment failures, and a model that outscored physicians on written cases did not make them better diagnosticians. A duty to consult would spread a model's errors as widely as its insights, and clinicians handed a biased model became markedly less accurate. Nor are such trials exotic: Taiwan's and Sweden's ran within ordinary hospital care and routine breast screening.
The companies that make these models are best placed to test the claim. Before anyone tells doctors they must consult a model, its makers should show, in randomized trials in ordinary care, that patients do better when doctors do: fewer serious diagnoses missed, fewer treatment failures, fewer deaths. Those trials should be registered before they start and published whatever they find. Health systems can add evidence of their own by giving a new tool first to a randomly chosen half of their clinicians, as Penda did, and counting what happens to patients.
Specialty societies should then weigh those trials task by task, as the gastroenterologists did for polyp software, and say plainly where consulting AI is expected, where it is a reasonable choice and where the evidence is too thin to say. When trials show that patients do better, the standard of care should move, and physicians with it. Until then, a model's answer is one more opinion for the physician to weigh, and the question to put to anyone selling one is what happened to the patients.
By Mr. Kennedy's account, Mr. Altman also said that two years ago he would not have advised anybody to take medical advice alone from AI. Opinions about these models have changed quickly. The evidence that patients do better when their doctors consult AI has moved more slowly, and it is the evidence that should set the standard.
- The summit remarksThe Make America Healthy Again Summit, a one-day event in Washington on Sept. 29, 2026, lists OpenAI and Anthropic among its supporters. In a conversation with Vice President JD Vance, Robert F. Kennedy Jr. said, "You have six minutes with a doctor today," and that AI "can give you a second opinion that is much better informed than any doctor in the country." He said Sam Altman, "over in our office a couple days ago," had said, "Two years ago, I wouldn't have advised anybody to take medical advice alone from AI," and that "it would be malpractice for a doctor to make a diagnosis or to make a prescription without at least checking AI." Medical Economics reported that Felipe Millon, head of OpenAI's Government Go to Market division, told attendees, "It is medical malpractice not to get a second opinion from AI today." Neither the transcript nor that report shows Mr. Kennedy or Mr. Millon citing a study.
- How language models in medicine have been studiedChen, Oermann and colleagues (Nature Medicine, March 2026; a systematic review assisted by a language model) identified 4,609 peer-reviewed studies of language models in clinical medicine published between January 2022 and September 2025. By their model's estimates, 1,048 (95% CI 847 to 1,252) used real-world patient data and most addressed simulated scenarios (1,857) or exam-style tasks (1,704); among the real-data studies, manual review confirmed 19 prospective randomized trials. They conclude that "rigorous, patient-centered evidence remains scarce." Funding came from the National Cancer Institute, the W.M. Keck Foundation and a Korean government grant; one author reports equity in OpenAI and another consulting for Google. An earlier review (Bedi and colleagues, JAMA, Jan. 28, 2025) of 519 studies published from January 2022 to February 2024 found that 5% used real patient care data; the commonest tasks were assessing medical knowledge, such as licensing examination questions (44.5%), and making diagnoses (19.5%).
- Google's AMIE, in simulation and in a clinicTu and colleagues (Nature, April 9, 2025) compared AMIE with 20 primary care physicians in a randomized, double-blind crossover study of text-based consultations with validated patient-actors, across 159 case scenarios in Canada, the United Kingdom and India. AMIE showed greater diagnostic accuracy and was rated superior on 30 of 32 axes by specialist physicians and on 25 of 26 by patient-actors; the authors note that text chat "is unfamiliar in clinical practice." In a single-site, single-arm study at Beth Israel Deaconess Medical Center's primary care clinic (The Lancet, Oct. 8, 2026), 114 patients enrolled and 98 completed both an AMIE interview and their urgent visit, with a physician monitoring every conversation in real time; none required a safety stop. Physicians found the AMIE output helpful for visit preparation in 33 of the 44 cases in which they reviewed it, and one rated an interaction somewhat harmful. Google says AMIE's differential diagnoses "matched the doctors' final diagnoses 90% of the time."
- Microsoft's and OpenAI's benchmarksNori, Horvitz and colleagues (arXiv preprint, June 27, 2025, revised July 2) report that Microsoft's MAI-DxO, paired with OpenAI's o3, reached 80% accuracy on 304 New England Journal of Medicine clinicopathological conference cases, and 85.5% with ensembling, against a 20% average for 21 generalist physicians practicing in the US or UK, who were told not to use search engines or language models; Microsoft says they worked without colleagues or textbooks. Microsoft calls the system a research demonstration "not yet approved for clinical use." OpenAI's HealthBench Professional (April 2026) has 525 physician-authored tasks drawn from clinicians testing ChatGPT for Clinicians, scored with rubrics written and adjudicated by physicians. ChatGPT for Clinicians (GPT-5.4) scored 59.0, base GPT-5.4 48.1, and specialty-matched physicians, writing with internet access, no time limit and no AI assistance, 43.7. Its authors write that "absolute benchmark scores are not directly comparable to real-world performance rates."
- Physicians with and without a model, on written casesGoh and colleagues (JAMA Network Open, Oct. 28, 2024; Stanford, Beth Israel Deaconess and other centers) randomized 50 physicians to GPT-4 plus conventional resources or conventional resources alone for up to six vignettes in an hour: median diagnostic reasoning scores were 76% and 74% (adjusted difference 2 percentage points, 95% CI -4 to 8; P=.60), and in a secondary exploratory analysis GPT-4 alone scored 16 percentage points higher than the conventional-resources group (P=.03). In a second trial (Nature Medicine, Feb. 5, 2025) of 92 physicians, those given GPT-4 scored 6.5 percentage points higher on five management vignettes (95% CI 2.7 to 10.2); GPT-4 alone did about as well as physicians using it. Among 457 hospitalist physicians, nurse practitioners and physician assistants reading vignettes of acute respiratory failure (Jabbour and colleagues, JAMA, December 2023), a deliberately biased chest X-ray model lowered diagnostic accuracy by 11.3 percentage points from a baseline of about 73%; a standard model raised it by 2.9 points.
- An AI assistant in Kenyan clinicsIn a quality improvement study reported July 22, 2025, with financial support from OpenAI, Penda Health gave its AI Consult assistant to a randomly selected half of its clinicians. Across 39,849 visits at 15 clinics, 108 independent physicians reviewing 5,666 visits found 16% fewer diagnostic and 13% fewer treatment errors with it; at an eight-day follow-up call, 3.8% of patients with it and 4.3% without were not feeling better (not statistically significant). A pragmatic cluster-randomized trial in 16 Penda facilities (Agweyu, Mateen and colleagues, Nature Medicine, June 26, 2026; funded by the Gates Foundation) enrolled 9,691 patients from April 22 to July 16, 2025. Primary outcome, treatment failure within 14 days: 102 of 4,693 (2.2%) with GPT-4o-based assistance, 94 of 4,654 (2.0%) without (adjusted odds ratio 0.77, 95% CI 0.55 to 1.08). Secondary outcome: in 2,000 encounters reviewed, appropriate diagnoses were recorded more often (adjusted odds ratio 1.74). The trial was "not powered to detect rare severe adverse events."
- Trials with patient outcomes: electrocardiograms and mammogramsLin and colleagues (Nature Medicine, April 29, 2024) ran a pragmatic randomized trial in Taiwan involving 39 physicians and 15,965 patients admitted through emergency or internal medicine departments, in which physicians received an AI report and warnings for patients whose electrocardiograms indicated a high risk of death. All-cause mortality within 90 days was 3.6% with the intervention and 4.3% without (hazard ratio 0.83, 95% CI 0.70 to 0.99), and the hazard ratio among high-risk patients was 0.69. Funding came from Taiwan's National Science and Technology Council and others. In Sweden's MASAI trial (The Lancet, January 2026), 105,915 women were randomly assigned to AI-supported screening or standard double reading. Interval cancer rates were 1.55 and 1.76 per 1,000 (82 and 93 cancers), a ratio of 0.88 that met non-inferiority and was not significantly different (p=0.41); sensitivity was 80.5% against 73.8%, specificity 98.5% in both arms, and screen-reading workload fell by 44%.
- Missed diagnoses, and software that finds moreNewman-Toker and colleagues (BMJ Quality & Safety) multiplied the incidence of 15 major vascular events, infections and cancers by literature-based rates of diagnostic error and harm, and estimated that 795,000 Americans (plausible range 598,000 to 1,023,000) die or are permanently disabled by diagnostic error each year: 371,000 deaths and 424,000 disabilities. The American Gastroenterological Association's living guideline on computer-aided detection in colonoscopy (Gastroenterology, 2025) drew on randomized trials with more than 30,000 participants: adenoma detection rates were 44.8% with the software and 37.4% without, gains largely from diminutive polyps of low malignant potential. Evidence on colorectal cancer incidence and mortality was of very low certainty; modeling suggested 11 fewer cancers and two fewer cancer deaths per 10,000 people over 10 years, against about 635 additional surveillance colonoscopies. The AGA made no recommendation for or against the software.