Bringing AI's answers to your doctor

This letter explains, once a week and in plain language, how artificial intelligence (AI) is showing up in health care and what you can ask about it. It gives general information, not medical advice. For questions about your own care, talk with your doctor.

Many people ask AI first

KFF, a health policy research group, asked adults in the United States about this in a poll this year. About a third said they had used an AI chatbot for health information in the past year. Of those, about two in three said a major reason was to get information quickly. About one in five said a major reason was that they could not afford to see a doctor or nurse. About as many said they had no regular doctor or could not get an appointment (KFF).

Doctors often don't hear about it from their patients. In a survey by the American Medical Association (AMA), 29% of doctors said not a single patient had told them about using AI. Most doctors don't mind: 70% said patients' use of AI for health information is positive or neutral when it is used appropriately (AMA).

What you can do: Tell your doctor or nurse at the start of the visit that you looked something up, so there is time to talk about it.

A chatbot depends on what you tell it

In a study led by researchers at the University of Oxford, AI chatbots were given written descriptions of ten medical problems. On their own, they named at least one of the right conditions about 95% of the time. But when ordinary people in Britain used the same chatbots in 2024, they named at least one of the right conditions less than 35% of the time. People who looked things up their usual way, mostly with an internet search, did better. Many left out details the chatbot needed. And when the chatbot did suggest a right answer, people often did not pick it (Nature Medicine).

OpenAI, the company that makes ChatGPT, says its health feature "is not intended for diagnosis or treatment" (OpenAI).

What you can do: Bring the whole conversation, not just the answer: what you typed and what the chatbot said. Then tell your doctor which possibility worries you most. That helps your doctor answer the question you really came with.

Don't let a chatbot tell you to wait

In January, researchers at Mount Sinai, a hospital system in New York, tested ChatGPT Health with made-up cases written by doctors. It sent textbook emergencies, such as stroke and severe allergic reactions, to the emergency room. The main test also included two true emergencies: a dangerous complication of diabetes, and an asthma attack in which breathing was about to fail. In about half of its answers about those two, it advised seeing a doctor within one to two days instead of going to the emergency room. In borderline cases, its advice was also more likely to change, usually to less urgent care, when a family member or friend in the story played down the symptoms (Nature Medicine). OpenAI said the test did not reflect how people typically use ChatGPT Health (NBC News).

For symptoms that are getting worse or seem serious, the study's authors advise getting medical care directly rather than relying only on a chatbot. Their examples are chest pain, shortness of breath, severe allergic reactions and changes in mental status. "In cases involving thoughts of self-harm, individuals should contact the 988 Suicide and Crisis Lifeline or go to an emergency department," Mount Sinai's announcement says (Mount Sinai). The AMA says AI tools "should never be used in emergencies" (AMA).

What you can do: If you think you might be having an emergency, call 911 or go to the nearest emergency room. With chest pain or other signs of a heart attack, call 911: it is almost always the fastest way to get treatment (American Heart Association). If you are thinking about hurting yourself, call or text 988 (988 Lifeline). Don't wait for a chatbot's answer, and don't let a reassuring one talk you out of getting help.

What a smartwatch alert means, and doesn't

In 2024, about four in ten U.S. adults said they had used a smartwatch, fitness tracker or similar device in the past year to track their health or activity (European Heart Journal: Digital Health). Much of what these devices show, such as sleep stages, "readiness" scores and heart rate variability, is sold as wellness information. The Food and Drug Administration (FDA) generally does not review those features the way it reviews medical devices (FDA).

Heart rate variability (HRV) is the slight change in time between heartbeats. What is normal for one person may not be normal for another, and it tends to drop with age. Cleveland Clinic says wrist devices "usually aren't sensitive enough to detect HRV accurately" (Cleveland Clinic).

Some features have been cleared by the FDA, such as Apple Watch alerts for possible high blood pressure (FDA) and for sleep apnea (FDA). They miss many cases. In Apple's own testing, the blood pressure alert caught about four in ten people who had high blood pressure (Apple). For sleep apnea, Apple says "the absence of a notification isn't intended to indicate the absence of sleep apnea" (Apple). In other words, no alert does not mean you don't have it. These alerts are not meant for people who already know they have high blood pressure or sleep apnea.

An alert is a reason to get checked, but it is not a diagnosis. At Mayo Clinic, researchers looked at 264 people who saw a doctor in 2018 and 2019 after an Apple Watch picked up an abnormal pulse. Thirty of them, about one in nine, were found to have a new heart rhythm problem that needed medical attention (JAMIA).

What you can do: If your watch sends an alert, make an appointment and bring the reading. If an alert comes with chest pain or trouble breathing, call 911. No alert does not mean everything is fine, so keep up with regular checkups. A family doctor writing on a blog of the American Academy of Family Physicians notes that the Apple Watch "is not a substitute for a home BP monitor" (AAFP). A BP monitor is a blood pressure cuff you can use at home. And if you feel unwell, don't wait for an alert.

Your privacy

HIPAA, the federal health privacy law, covers health plans and most doctors and hospitals. It generally does not cover a chatbot or app you choose to use on your own (HHS). In KFF's poll, about four in ten people who used AI for health said they had uploaded personal medical information, such as test results or doctor's notes (KFF).

What you can ask: Before you upload test results or notes, read the company's privacy policy. Does it keep what you share? Can you delete it? Does it use your information to train its AI?

A special topic for doctors published with this letter covers the same subject: The patient's AI got there first.