The promise of Artificial Intelligence (AI) in medical research has always been the acceleration of discovery: the ability to sift through millions of pages of data to find correlations that the human mind would fail to detect. However, a recent alarming report, highlighted by CBS News, brings to light a dark side of this technology. Researchers have found that Large Language Models (LLMs), such as ChatGPT and Gemini, often "fabricate" bibliographic citations that appear perfectly convincing but do not actually exist.
This trend, known in the industry as "hallucinations," takes a dangerous turn when applied to the biomedical field. In science, a bibliographic citation is not just a formal procedure; it is the foundation upon which knowledge is built. If a study is based on data that never existed, the entire edifice of medical care risks collapsing.
The Mechanism of the Illusion
To understand why this happens, we must understand the nature of AI models. LLMs are not databases or truth-seeking machines; they are statistical word-prediction engines. They are trained to recognize patterns in language. When a researcher asks the AI to document a claim, the model "knows" what a medical citation looks like: it has an author's name, a date, a journal title, and a DOI number. If it cannot find an actual reference, it often synthesizes a new one that fits the context of the discussion.
The problem is that these fabricated citations are extremely plausible. They use the names of real professors from top universities and titles of prestigious journals like *The Lancet* or the *New England Journal of Medicine*. In one study that examined 115 citations produced by a popular AI platform, nearly 47% were partially or entirely fabricated. This error rate is unacceptable in a field where precision saves lives.
Risks to Public Health
The integration of AI into scientific writing has created a new crisis in scientific ethics. Researchers, under the pressure of "publish or perish," may use these tools to speed up their work, omitting the rigorous scrutiny of sources. If a false citation passes through the peer review process—which is already under pressure due to the volume of papers—then misinformation enters the official body of medical knowledge.
Imagine a clinician reading a review of a new cancer treatment supported by supposedly "robust studies." If these studies are products of an AI's imagination, the doctor may make decisions that harm the patient. Furthermore, there is the risk of "circular error": future AI models will be trained on texts that already contain false citations, creating a vicious cycle of misinformation that will be impossible to clean up.
Toward a Solution: Technology and Human Vigilance
The solution is not to ban AI, but to radically change how it is used. Techniques such as RAG (Retrieval-Augmented Generation) are already being developed, where the AI is forced to draw information only from a specific, verified database (such as PubMed) instead of relying on its internal parameters. However, even these methods are not infallible.
The scientific community must set stricter rules. Journals must require the disclosure of AI use and use AI tools themselves to verify citations. Above all, the responsibility remains with the human researcher. AI can be an excellent drafting assistant, but it should never be considered a source of truth. The crisis in biomedical literature is a reminder that in the age of automation, critical thinking and meticulous verification are more valuable than ever.
"AI does not lie in the human sense; it simply fails to distinguish the difference between a probable sentence and a factual reality. In an operating room or a laboratory, this distinction is the difference between life and death."
