Why Do Hospital AI Pilots Fail to Scale? Plan for Day Two
Hospital AI pilots usually fail to scale because nobody planned for what happens after the pilot succeeds. Paul Uhrig, a healthcare lawyer and former chief legal and digital health officer at Bassett Healthcare Network, told Lori Ellis on Open Door Salon that a pilot can meet every one of its own criteria and still leave the health system with no workflow, resources or change management to roll it out. Eddie Qureshi, founder and CEO of Rainfall Health, added the vendor’s side: an AI that is not one of an executive’s top priorities will not be bought, however good it is.
Why do successful pilots stall?
Because the end game was never part of the plan. Uhrig said the health systems that implement well think about scale from the first conversation:
People often talk about a pilot and then don’t really think about, okay, if the pilot is successful, what is the workflow required to roll it out at scale?
He listed what that question covers: the resources required, the change management required, and where the blockers will be.
Who runs a pilot, and why does that matter?
The enthusiasts, which flatters the result. Uhrig explained:
Pilots usually start with the evangelist ... When you go at scale, you might start encountering more resistance from people.
A pilot staffed by volunteers measures how a tool performs with the people most willing to make it work. Scale brings in everyone else. One clinical tool that did stick is covered in our post on what ambient listening in healthcare is and why clinicians adopted it.
Who is frustrated when a pilot cannot scale?
Both sides of the contract. Uhrig said the gap between pilot success and rollout is what frustrates people most:
It frustrates the health system. It frustrates the vendor who is selling that to them.
The pattern is familiar outside hospitals. Our earlier post on why government AI projects fail describes temporary funding that launches a program nothing pays to sustain.
What should executives ask before a pilot starts?
What problem it solves, and how anyone will know. Uhrig said the primary questions start with return on investment:
What is the problem that is being solved?
Then comes measurement, which he said is the whole point of the purchase:
Because that is what we are all looking for, it’s improvement, whether it’s efficiency, whether it’s in outcomes.
After that, change management, not only for clinicians when the tool is clinical but for the wider workforce and for patients. The data questions come last on his list but not in importance, since security was where the conversation began:
Is it being used to train the models? Should it be used? What are the constraints?
Does a narrow AI product scale better than a broad one?
In Qureshi’s experience, yes. His company started broad, working on rural health with Indigenous communities, veteran groups and Air Force service members, and narrowed as it matured, which he said runs the opposite way to what you would expect:
Healthcare is such a complex problem we got narrower.
Rainfall Health now works only on managing episodes of care for Medicare patients. He described his entire market as roughly the hospitals in one Medicare model and about 1,500 executives, and said that within that space the deals close quickly once the product connects.
Why does priority decide whether AI gets bought?
Because an executive can only carry a few. Qureshi said that only two or three priorities ever surface for any member of the C-suite, and that this decides more than the technology does:
So if you’re not one of those top three priorities, no matter how great your AI is, it’s just not the right solution.
He said his company’s sales cycles now average under three weeks, which he attributes to solving one specific near-term problem and proving its return. He compared asking whether a health system wants AI to asking whether someone wants the internet:
There’s five websites that people spend 80% of their time on, right? There’s few use cases that are immediate.
How can vendors and health systems share the risk?
By tying payment to value. (Uhrig advises Rainfall Health.) Uhrig said that in his years inside a health system, peers at rural and urban systems alike were solving similar problems, and Qureshi said the struggle looks the same from the vendor side too. His answer to it:
Let’s align incentives. If I don’t help provide value to you as a health system with my AI, well, you don’t have to pay me.
He framed it as sustainability: a health system has to stay in business, because if it is gone, its whole community is at risk. Patients are already bringing their own AI into this picture, as our earlier post on how AI is changing doctor-patient conversations describes, and the Medicare context for Qureshi’s company is in what the CMS TEAM model is.
Companies that want to reach the leaders who follow this conversation can learn about sponsoring Open Door Salon.
This post is drawn from the recorded, on-the-record conversation between Lori Ellis, Paul Uhrig and Eddie Qureshi on Open Door Salon.