AI sounds sure either way. Check it before it sticks.

Ask an AI tool a medical question and you get a clear, well-organized answer in seconds. That is the appeal. It is also the risk, because a wrong answer arrives in exactly the same tone as a right one. For a student about to turn that answer into flashcards, a wrong fact stops being a small error. It becomes something you rehearse until it feels true.

Why wrong answers go down so easily

In 2023, Ziwei Ji and colleagues published a survey of research on hallucination in natural language generation. Their starting point is that text generation has become far more fluent and coherent, and that it is also prone to hallucinate unintended text. Fluency and accuracy are separate properties. A system can improve at one without the other.

People are not well set up to catch this. In a 1999 study, Rolf Reber and Norbert Schwarz showed participants statements printed in colors that made them easy or hard to read, and asked them to judge whether each was true. Statements that were easy to see were judged true more often than chance, while moderately visible ones were judged at chance. Ease of processing nudged judgments of truth. AI text is built to read smoothly, so it is reasonable to worry that the same pull applies. That is an inference from the study, not a finding about AI, but it is a good reason not to trust your first impression of an answer.

This is not hypothetical in medicine

In 2023, Jesutofunmi Omiye and colleagues asked four commercially available language models questions about race-based medical misconceptions, drawn from physician experts and from earlier work on misconceptions believed by medical trainees. All of the models had examples of perpetuating race-based medicine in their responses. The models were also not always consistent when the same question was asked repeatedly.

Two lessons follow. A clean-looking answer can still carry content that medicine has moved away from. And one answer is a sample, not a verdict: ask again and you may get something different.

A routine that fits in five minutes

Clinicians are already taught that their own reasoning can fail. Pat Croskerry's 2003 review in Academic Medicine describes cognitive errors in diagnosis, including overconfidence and anchoring, and argues for deliberately debiasing our thinking. Checking an AI answer is the same discipline applied to a new source. Here is a routine that works on any tool.

• Break the answer into claims. Number the distinct facts: mechanisms, associations, doses, thresholds, first-line choices. You cannot check a paragraph, but you can check a list.
• Ask how sure it is. Have the AI label each claim by confidence and tell you what to double-check, then treat that as a clue about where to look. A model's confidence is not a source.
• Flag the high-risk facts. Doses, units, lab thresholds, contraindications, interactions, age and weight cutoffs, and any claim with always or never in it get checked every time, whatever confidence the AI reports.
• Argue against it. Ask for the strongest case that the answer is oversimplified, out of date, or missing an exception such as pregnancy, kidney disease, children, or older adults.
• Check the flagged items in a source you trust: the current guideline, the prescribing information, or your course materials. If the AI names a study or guideline, confirm that it exists and says what was claimed before you rely on it.

Run the check in a fresh chat, ideally in a different tool, so the second look is not anchored on the first answer. Then make a decision: study it as is, study it after verifying the flagged items, or set it aside. If your course materials disagree with the AI, your course materials win.

The goal is not distrust

AI can explain, quiz, and organize faster than any single person. The aim is to use it the way you would use a knowledgeable colleague who is sometimes wrong and does not always sound it: gratefully, and with the book open.