AI Clinical Decision Support: The Trust Problem

I keep thinking about why AI still can’t change a doctor’s mind.

Not because the models are bad. Some of them are genuinely impressive.

But impressive models and useful tools are not the same thing.

Health systems invest millions in clinical decision support, deploy it inside the EHR, and then watch utilization rates flatline. Clinicians click through alerts without reading them. Recommendations get ignored. And the vendor gets blamed.

But the vendor is not always the problem.

A 2025 study published in the Journal of the American Medical Informatics Association found that physicians override clinical decision support alerts at rates exceeding 90% in some health systems. That is not a technology failure. It is a trust failure.

Why?

Three reasons nobody wants to say out loud.

🔍 First, most clinical AI is still a black box. Physicians are trained to understand causality. Show a doctor an alert that says “high sepsis risk” without explaining why, and many will dismiss it. Not because they are resistant to AI, but because good clinical training teaches skepticism without evidence.

Second, alert fatigue is structural, not behavioral. Primary care physicians receive dozens of alerts every day. Research has shown that when health systems reduce unnecessary alerts, meaningful responses increase. More alerts do not create more safety. They create more noise.

Third, many AI tools were built for compliance and billing workflows, not clinical thinking. When AI is buried inside the same interface physicians already struggle with, even accurate recommendations are easy to ignore.

What actually works?

The evidence points to specificity. Narrow, high-confidence alerts paired with a clear next action consistently outperform broad notification systems.

AI that briefly explains its reasoning also performs better. Explainability is not just a regulatory checkbox. It is a clinical adoption strategy.

And integration matters as much as accuracy. A tool that is 95% accurate but buried three clicks deep will often underperform one that is slightly less accurate but appears naturally in the clinician’s workflow.

I have watched this pattern play out firsthand. At Oatmeal Health, we think constantly about what makes a clinician actually act on a finding. The answer is almost never more data. It is better context, delivered at the right time and in the right workflow.

Health systems will not solve AI adoption by buying newer models. They will solve it by redesigning the interaction between the tool and the human.

The physician is not the obstacle. The physician is the user.

We keep building AI for the system. We need to start building it for the person standing in front of the patient.

If you lead a health system or work in clinical informatics, what has actually improved AI adoption in your organization?

👉 Follow me for daily healthcare insights and updates on LinkedIn.

🔍 For deeper analysis, subscribe to my Substack, where I share long-form articles and perspectives on healthcare, AI, diagnostics, and the future of care.