Running up the tab

Twenty general-purpose AI chatbots were given 200 anonymized clinic and emergency department notes, cut off just before the doctor's assessment, and asked what should happen next. Priced at Medicare's rates, the tests and referrals they proposed came to an average of $314 a visit. The physicians' own documented plans came to $97. At two-thirds of the visits the doctors ordered no tests at all, and every system in the study suggested some, usually the same stock panel.

That study, published in NEJM AI, arrived in a week when Medicare decided to pay doctors less for their time and took another step toward a way of paying for software. The case for paying for clinical AI is that it saves time and money. The week's evidence suggests that, in ordinary use, it often spends both, moving the cost into extra tests, slower inboxes and errors that someone must catch. Medicare, insurers and hospitals should pay for clinical software only when it shows, in practice rather than in a sales presentation, that it gives back what it promises.

Start with the money. Under the final 2027 fee schedule, issued on November 4, the conversion factor, the dollar figure that turns each service's relative value into a fee, falls by 2.31% for most doctors, a deeper cut than it proposed in July, chiefly because a one-year 2.5% increase Congress granted for 2026 expires. The same rule pays less when an office visit and a minor procedure fall on the same day. The companion rule for hospital outpatient care renames what Medicare called software as a service, now "Software as a Medical Service," and places those services in New Technology payment groups as "an interim step toward a broader payment policy." The permanent policy, in other words, is still to be written.

The evidence on time is less flattering than the brochures. At Columbia University Irving Medical Center, researchers examined nearly a million replies to patient-portal messages from more than 1,500 clinicians, before and after Epic's AI draft-reply tool was switched on. A message answered with the draft went out faster. Yet adopting the tool was associated with replies about 9% slower overall, and with no overall drop in after-hours work. The authors' explanation is the one any doctor would guess: every draft must be read, and a poor one must be fixed or thrown away.

Then there are the errors that someone has to catch. Doximity, which sells a dose-checking product of its own, tested nine widely used AI models on more than 8,500 dosing questions and reported that about one answer in 50 contained a serious or potentially lethal mistake, most often a failure to adjust for age, kidney function or drug interactions that the question had supplied. Those are the company's own numbers, released alongside its product, and deserve the discount that implies. Doctors are using such tools anyway. STAT reported that Monument Health, in South Dakota and Wyoming, blocked the free clinical chatbot OpenEvidence this year; one of the objections came from a "super user" who had stored more than a hundred patient records in it and used it to help make medical decisions.

Patients are getting the same tools, with fewer guardrails. On November 2 athenahealth, which supplies record systems to physician practices, let patients connect their records to Anthropic's Claude chatbot, which can then consult their lab results, medications, problem lists and imaging reports when they ask a question. A company executive told GovInfoSecurity that because the sharing is directed by the patient, "HIPAA is not applicable in this use case." Whatever the chatbot makes of those records will arrive at the next appointment, and the time spent sorting good answers from bad ones will be the physician's, at next year's lower rate.

Perhaps the chatbots are simply more thorough than doctors with fifteen minutes and a full waiting room. The study measured what the extra tests would cost, not whether patients needed them, and some may have been warranted. Advocates can also point to a randomized trial in Dresden, published in NEJM AI, in which patients sent an AI-written, quality-checked plain-language summary of their findings before a neurosurgery visit rated the encounter higher and spent a median of 25 minutes in it rather than 33. Even the costly chatbots improved when told to be sparing but safe. These points have force, but they make the case for paying on results. The Dresden gain was found by counting, in a trial, and the chatbots' restraint had to be asked for. A payment system that rewards software for being used, rather than for what it saves, will get neither.

Medicare should write its permanent software policy so that a product earns payment by showing, in ordinary practice, fewer tests, fewer errors or less clinician time, and it should count the minutes doctors spend checking the output as a cost. Hospitals and medical staffs should measure inbox turnaround, after-hours work and test ordering before and after any AI tool goes live, and switch off those that make the numbers worse. Health systems should give doctors a vetted clinical chatbot under a contract that covers patient data, rather than leave them to free ones. Federal regulators should say what protection, if any, follows a patient's records once they leave the portal for a chatbot. Physicians should treat a chatbot's work-up as a list to prune rather than a plan, and should ask patients which chatbot they have already consulted.

Told to be sparing, the chatbots in the NEJM AI study cut most of their excess. Medicare should give the companies that make them the same instruction, in the only language every vendor understands: payment for what they save, and nothing for what they spend.