Built for the buyer
Much clinical AI is designed for the health systems that pay for it. The doctors who use it should get a vote before it is bought and an off switch for when it slows care.
Listen to this special topic
read by an AI voiceThe American Medical Association warned in October that doctors may answer for software they never chose. Its brief on liability for clinical AI says physicians “may be expected to rely on AI systems they were not informed were in use or had no authority to select or decline.” It wants them free to refuse unproven tools without being penalized. It also says the AI duties of hospitals deploying such tools are still unsettled.
The trouble starts long before anyone is sued. A doctor may use a clinical AI tool all day, but most of the money spent last year on generative AI built for health care came from health systems. Much of that software is built around what the buyer wants, a clean claim, a full schedule and a better margin. The doctor's day comes second. Physicians should get a formal vote before a clinical AI tool is bought, and a way to switch it off when it slows care.
Ask doctors what they want and they describe their chores. When Doximity, which has AI tools of its own, surveyed its physician members, more than four in five wanted help searching the literature, writing letters for patients, educating patients and handling insurance paperwork. Their commonest worry was accuracy. In the AMA's survey early this year, the top requirements for adopting AI included proof of safety and efficacy from a trusted body and “a designated feedback channel when issues arise.” Most wanted a direct role. In all, 85% wanted to be consulted on bringing AI into their practice, or to be responsible for it.
What gets bought looks different. Menlo Ventures, a venture firm that invests in health care AI, surveyed about 700 health care executives last year. It estimated that the biggest spending category was scribes, the software that listens to a visit and drafts the note. Next came automated coding and billing. The firm says scribes reduce burnout and coding software recovers revenue lost to errors and denials. Health systems lead, it wrote, because their need is greatest and the return obvious. In 2024 Microsoft said physicians at Northwestern Medicine who used its scribe in at least half their visits could see about 11 more patients a month on average, which it presented as better access to care.
Some products are built around improving the bottom line. Ambience Healthcare markets a scribe with “coding awareness as a core feature.” It surfaces billing codes for the clinician to review at the point of care, and the company claims an average of $4.50 more revenue per visit. Ambience presents this as compliance and accurate coding. Perhaps it is. The review still falls to the doctor, in the doctor's time. Scribes change the visit in other ways. Dina Capalongo, a primary care physician at Penn, told KFF Health News that she now says aloud what she is doing and finding during a physical exam, so the AI scribe can record it. Patients find it very interesting, she said. Genevieve Melton-Meaux, chief health informatics and AI officer at Fairview Health Services, said that for a frightened patient, hearing worrying things during an uncomfortable exam “does not help the situation and honestly is insensitive to what the patient is going through.”
Only 8% of physicians in Doximity's survey said their institution's process for deciding on AI was clear and that they understood its policies. Nearly half called it “still evolving.” Health systems do switch tools off. Baptist Health South Florida retired a virtual care navigator that fewer than 100 patients a month used, and that suggested urgent or emergency care in nine cases out of ten when better options existed. The question is who pulls the switch, and on what evidence.
Health systems have a fair answer. They pay for these tools, sign the contracts and carry the risk on thin margins. Their biggest purchase is the scribe, which doctors wanted. Robert Wachter, chair of medicine at the University of California, San Francisco, told KFF Health News that many systems that might once have run a hard-nosed calculation of return on investment are softening, and “realizing that the cost of recruiting and retaining doctors is pretty high.” Doctors in the AMA's survey worried less about privacy with tools their institution supplied. A vote on every purchase sounds like one more committee. Where buyer and doctor want the same thing, the vote will pass in one meeting. It matters where they part, as with billing prompts inside the visit or a scribe sold on extra patients a month. And some problems appear only in use, as Baptist Health's navigator showed. That is why the switch matters as much as the vote.
Medical staffs that have not done so should write both into their bylaws. A clinical AI tool should go live only after the physicians who will use it have seen its evidence and approved it, much as a hospital evaluates a drug for its formulary. The AMA's brief leaves selection to those who deploy a tool, though its House of Delegates has said since 2024 that the organized medical staff should be “an integral part at the outset of choosing” such tools in hospital care. Physicians should share it. The contract should then let the medical staff turn the tool off without penalty if, after a trial in the physicians' own clinics, it misses targets set in advance. Minutes per note, time to answer a patient and errors in a sample of charts would do. The brief already asks deployers for processes to discontinue a tool “when warranted.” The doctors using it should help decide when that is.
The AMA's brief warns of doctors relying on software they had “no authority to select or decline.” They still sign the note. A vote before the purchase and a switch after it would give them a say in the tools that write it. Hospitals can start on both now.
- The AMA's brief on liability for clinical AIThe American Medical Association's Issue Brief: Liability for Clinical AI is dated October 2026. It is a policy brief, not a study; the AMA announced it on Oct. 29, 2026. It says physicians may be held responsible for errors they cannot reasonably detect, and may be expected to rely on AI systems they were not informed were in use or had no authority to select or decline. It says physicians should be free to decline unproven AI, and should retain the authority to decline AI systems whose clinical validation, usefulness or place in the standard of care remains uncertain, without being penalized for exercising their clinical judgment. It names health systems, hospitals, physician practices and other organizations as deployers, says their AI-specific responsibilities remain unsettled, and says they should have processes for responding to adverse events and for modifying, restricting or discontinuing use when warranted. Separately, AMA policy H-225.940, Augmented Intelligence and Organized Medical Staff, adopted by the House of Delegates in 2024, says organized medical staff should be an integral part at the outset of choosing AI tools in hospital care.
- The AMA's 2026 physician surveyThe AMA's 2026 Physician Survey on Augmented Intelligence (March 2026) was fielded from Jan. 15 to Feb. 2, 2026; a total of 1,692 U.S. physicians participated. 85% want to be consulted or be responsible for adoption of AI into their practice: 55% prefer to be consulted and 30% want responsibility for implementation. The top two factors physicians said would facilitate adoption were data privacy assurances (86% say important) and validation of safety and efficacy (88% say important). The report lists the top requirements as validated safety and efficacy from a trusted entity, strong data privacy protections, a designated feedback channel when issues arise, coverage under standard malpractice insurance and EHR integration. 71% were somewhat or very concerned about patient privacy with AI tools not provided or sponsored by their institution, against 42% for institutional tools.
- Doximity's 2026 AI reportDoximity surveyed its physician members in March and April 2025 and again from November 2025 to January 2026; 3,151 U.S. physicians in 15 specialties completed the survey. Participation did not require prior experience with AI or use of Doximity's AI tools, and the report says its demographics are not population-based and findings may not be generalizable. Among all physicians surveyed, interest was strongest in literature search (87%), drafting patient support letters (86%), patient education (84%) and insurance correspondence such as prior authorizations (82%); ambient scribe technology drew 79%. The most-cited concern was the accuracy and reliability of AI-generated outputs (71%). 47% said the AI decision-making process at their institution is “still evolving”; 8% said the process is clear and they understand the policies and guidelines.
- Where health care's AI money goesMenlo Ventures, a venture capital firm that says it invests in health care AI, published 2025, The State of AI in Healthcare, on October 21, 2025. Its estimates rest on a Morning Consult survey of 700+ executives (N=700 in a footnote) in August and September 2025 and a bottoms-up revenue analysis. It puts 2025 spending on health-care-specific generative AI at $1.4 billion, with health systems supplying $1 billion (75%), outpatient providers $280 million (20%) and payers $50 million (5%). Discussing that total, it puts ambient clinical documentation at $600 million, which it says reduces physician burnout, and coding and billing automation at $450 million, which it says recovers revenue lost to coding errors and denials. Becker's Hospital Review (Oct. 28, 2025), citing the report, lists health systems' own AI spending: ambient scribes $475 million, coding and billing $400 million, patient engagement $70 million, and payer engagement and prior authorization $50 million each. Those add up to about the $1 billion health systems supplied, so the two sets cover different buyers and do not conflict.
- What two vendors say their scribes deliverRobert Dahdah, corporate vice president at Microsoft, wrote on September 26, 2024 that DAX Copilot was helping Northwestern Medicine in Chicago improve patient access to care, with physicians using it in at least 50% of patient encounters and able to see an average of 11.3 additional patients per month, and that users there reported spending an average of 24% less time on notes. Ambience Healthcare's product page for ambulatory clinical documentation integrity says its scribe surfaces codes for clinicians to review at the point of care, recommends the evaluation and management level with modifier-25 during wellness visits, which it says boosts reimbursement without disrupting clinician workflows, and identifies diagnoses tied to risk adjustment. It lists a 10% uplift in HCC capture and a $4.50 average increase in revenue per E/M visit, and presents the product as improving compliance and coding accuracy. These are the companies' own figures; neither page, as read, cites a study, sample or period for them.
- Scribes in the exam roomKFF Health News (Michelle Andrews, Jan. 27, 2026) reported that some clinicians say AI scribes are reshaping encounters. Dina Capalongo, a primary care physician at Penn Internal Medicine in Media, Pennsylvania, said she now narrates her physical exams so the scribe can capture them, and that people find this very interesting. Genevieve Melton-Meaux, a University of Minnesota professor in the Division of Colon and Rectal Surgery and chief health informatics and AI officer at Fairview Health Services, said that saying things anxious patients may worry about during an uncomfortable examination does not help, and that she keeps such points in mind and makes sure to record them after the visit. Robert Wachter, chair of the Department of Medicine at the University of California-San Francisco, said many health systems that might initially have done a hard-nosed return-on-investment calculation are softening on it, realizing that recruiting and retaining doctors is costly.
- Tools health systems retired in 2025Becker's Hospital Review (Giles Bruce, Dec. 30, 2025) reported that FMOL Health in Baton Rouge, Louisiana, retired a professional billing tool meant to ease provider coding and ended the contract before go-live; its senior vice president and chief information officer, Will Landry, said the implementation was far more complex than the system had been told. Baptist Health South Florida dropped an AI-powered virtual care navigator, according to its chief digital and information officer, Sha Edathumparampil; Becker's reports that fewer than 100 patients engaged with it monthly and that in 90% of cases it suggested urgent or emergency care when more suitable options were available.
- How hospitals review drugsThe ASHP Guidelines on the Pharmacy and Therapeutics Committee and the Formulary System (Ciccarello and colleagues, American Journal of Health-System Pharmacy, 2021, volume 78, issue 10, pages 907 to 918), listed by the federal ASPR TRACIE resource center, cover formulary system management for hospitals and health systems, the organization and operation of the pharmacy and therapeutics committee, and the evaluation of medications for formularies.