Can AI Language Models Learn When to Avoid Unnecessary Medical Tests?
A new study by Israeli researchers tests AI language models on avoiding unnecessary medical tests. The findings show models rarely miss critical interventions but occasionally recommend excess procedures where human clinical consensus is lacking.
A new study led by Dr. Dean Ariel from Clalit Health Services' Tel Aviv district, Dr. Guy Leven from Ben-Gurion University of the Negev, and Dr. Stav Endelman from Rabin Medical Center has evaluated whether artificial intelligence language models know when to avoid unnecessary medical procedures. The research explored the 'Choosing Wisely' initiative, testing hundreds of scenarios across six models from Google, Anthropic, and OpenAI.
While medical overtesting often leads to anxiety, wasted resources, and unnecessary complications, undertesting poses its own risks. The study found that language models recommended avoiding necessary interventions in only 0.6% of cases, while suggesting unnecessary interventions in 5.4% of cases. Most errors occurred in areas where human medical professionals themselves face clinical controversy, such as thyroid ultrasounds for small incidental nodules and routine ECGs for low-risk patients.
Researchers emphasized that true medical AI integration requires balancing active diagnosis with restraint. The findings suggest that while modern language models generally understand clinical boundaries, they still mirror human dilemmas when facing ambiguous guidelines and patient anxieties.
