11 hours ago
https://www.eurekalert.org/news-releases/1151505
EXCERPTS: As people increasingly turn to artificial intelligence (AI) for health information, a new study by researchers at the Icahn School of Medicine at Mount Sinai raises questions about what happens when commercial advertising appears alongside a clinical question. The findings were reported in the November 11 online issue of The Lancet Digital Health: https://www.thelancet.com/journals/landi...8/fulltext.
Researchers tested 12 AI models from major technology companies, including Google, OpenAI, and Anthropic, to determine whether a clearly labeled pharmaceutical advertisement could influence which treatment an AI model recommended. They found that when the AI models were given a physician-written clinical case, preceded by an ad for a drug, the ad could shift which treatment the AI recommended, making it more likely to select the advertised drug.
Across 33 clinical scenarios, the advertised drug was selected in 47.6 percent of responses when an ad was present, compared with 35 percent without an ad. In four cases, models shifted from selecting the advertised drug zero percent of the time to selecting it 100 percent of the time when the ad was introduced.
“What I find quite interesting, and even surprising, was how reasonable the answers often appeared,” says physician-scientist and first author Mahmud Omar, MD, a lecturer in the Windreich Department of Artificial Intelligence and Human Health at the Icahn School of Medicine, who leads research on the safety, reliability, and real-world effects of generative AI in clinical care. “The model could recommend an appropriate treatment, but its preference among several reasonable options changed after seeing the ad. Because the final answer could still appear medically sound, that influence could be easy to miss in typical research evaluations and in other patient and physician facing personal use cases.”
[...] The findings do not mean pharmaceutical advertising will necessarily cause AI models to recommend unsafe or inappropriate treatments, say the investigators. In the study, advertisements did not clearly increase recommendations for unsupported supplements or clearly suboptimal drugs. The researchers also did not examine whether changes in AI recommendations ultimately affected prescriptions, patient decisions, costs, or health outcomes.
The researchers say the findings point to several practical steps for AI developers and health systems, including testing AI systems with and without promotional content to determine whether recommendations change, clearly disclosing when advertising is presented to an AI system, and keeping promotional material separate from clinical advice.
The findings may also have implications for regulators as they consider how existing rules governing pharmaceutical advertising apply when promotional content is incorporated into AI-generated health information... (MORE - missing details, no ads)
EXCERPTS: As people increasingly turn to artificial intelligence (AI) for health information, a new study by researchers at the Icahn School of Medicine at Mount Sinai raises questions about what happens when commercial advertising appears alongside a clinical question. The findings were reported in the November 11 online issue of The Lancet Digital Health: https://www.thelancet.com/journals/landi...8/fulltext.
Researchers tested 12 AI models from major technology companies, including Google, OpenAI, and Anthropic, to determine whether a clearly labeled pharmaceutical advertisement could influence which treatment an AI model recommended. They found that when the AI models were given a physician-written clinical case, preceded by an ad for a drug, the ad could shift which treatment the AI recommended, making it more likely to select the advertised drug.
Across 33 clinical scenarios, the advertised drug was selected in 47.6 percent of responses when an ad was present, compared with 35 percent without an ad. In four cases, models shifted from selecting the advertised drug zero percent of the time to selecting it 100 percent of the time when the ad was introduced.
“What I find quite interesting, and even surprising, was how reasonable the answers often appeared,” says physician-scientist and first author Mahmud Omar, MD, a lecturer in the Windreich Department of Artificial Intelligence and Human Health at the Icahn School of Medicine, who leads research on the safety, reliability, and real-world effects of generative AI in clinical care. “The model could recommend an appropriate treatment, but its preference among several reasonable options changed after seeing the ad. Because the final answer could still appear medically sound, that influence could be easy to miss in typical research evaluations and in other patient and physician facing personal use cases.”
[...] The findings do not mean pharmaceutical advertising will necessarily cause AI models to recommend unsafe or inappropriate treatments, say the investigators. In the study, advertisements did not clearly increase recommendations for unsupported supplements or clearly suboptimal drugs. The researchers also did not examine whether changes in AI recommendations ultimately affected prescriptions, patient decisions, costs, or health outcomes.
The researchers say the findings point to several practical steps for AI developers and health systems, including testing AI systems with and without promotional content to determine whether recommendations change, clearly disclosing when advertising is presented to an AI system, and keeping promotional material separate from clinical advice.
The findings may also have implications for regulators as they consider how existing rules governing pharmaceutical advertising apply when promotional content is incorporated into AI-generated health information... (MORE - missing details, no ads)