Almost every healthcare AI pilot I have seen clear its success criteria has gone on to struggle in production, sometimes within months of going live, and almost always for reasons that had nothing to do with the AI model itself. The pilot is not a useful predictor of what happens when a tool actually has to run inside the hospital’s operational environment, and we keep treating it as one.
Pilots succeed in part because they tolerate the kinds of architectural shortcuts that production environments, with their actual call volumes and data freshness requirements, simply cannot. A one-off data extract for the vendor. A custom integration someone on the team built over a weekend. A nightly refresh job that runs because nobody wanted to spend three months negotiating API quotas with the EHR vendor. The clinical champion gets a clean demo, the budget owner gets a believable ROI number, somebody signs off on the production rollout. Eighteen months later, with three or four such tools running side by side, the shortcuts collide. The integration team is on call most weeks. The API quotas blow up at unpredictable times. The clinicians have started to notice that two AI tools sometimes disagree about the same patient. What stalls these strategies is not the AI.
Healthcare AI spending hit roughly $1.5 billion in 2025. EHR vendors are shipping AI features by the dozens. Buying has gotten ahead of architecture, and we are now procuring AI the same way the industry procured EHRs a decade ago: one decision at a time, by separate budget owners, with no shared data layer or governance posture holding the pieces together. The integration debt this is producing will look familiar to anyone who survived the EHR implementation wave. What is different this time is that each AI tool both reads from and writes to the same data the next tool depends on, so the debt accumulates faster than anybody in finance is modelling, and the cleanup is harder because each tool now depends on the others.
Three patterns keep showing up, and each is much cheaper to prevent than to fix later.
The data-on-demand fallacy. AI vendors will tell you their tool works with your existing data, and at pilot scale that is genuinely true. The trouble starts around the third or fourth tool. By the time you are running three or four of these in production, each vendor has quietly built its own pipeline against the EHR, with its own extraction logic and field mappings and refresh schedule, and your integration team is now responsible for maintaining all of them indefinitely. The EHR’s API call quotas start spiking at unpredictable times. The data each tool sees has begun to drift, so the population-health platform believes a patient has been on metformin for two years while the ambient scribe thinks the same patient was just started on it last week. Neither tool is wrong, technically. They are pulling from different snapshots of the same chart.
There is no clever engineering trick that fixes this. The only honest answer is a curated, governed data layer, typically FHIR-native, that sits in front of the EHR, and every AI tool reads from it instead of building its own bypass. Most health systems have not made that commitment yet, and the longer they wait, the more bespoke pipelines they will have to migrate when they finally do.
The governance lag. Awareness of AI governance has risen sharply in healthcare over the past year. In practice, very little has actually changed. Most health systems can list the AI tools they use. Far fewer can answer the questions that matter. Who owns the accountability when an ambient scribe transcribes the wrong medication dose? Who approves the specific model version a prior-authorization tool is using on any given Tuesday? How do you reconstruct an AI-generated output if CMS comes back asking about it eighteen months later?
What this means for architecture is that governance can’t live inside each application. It has to live one layer down, in the data and integration plane that all the applications share. Model versioning, lineage tracking, audit logging, human-review checkpoints, these need to sit at the platform level rather than inside each tool. When governance gets retrofitted onto applications after the fact, every vendor’s model update breaks something, and the CMS audit findings start to pile up.
The agentic ceiling. For most health systems, generative AI is now a solved buying problem. You pick a vendor, run a pilot, roll it out. Agentic AI is a different conversation. Systems that take action against the chart, rather than just reading from it, need real-time integration. They need transactional reliability. They need authorization boundaries that hold when the system is under load, and audit trails that explain every action the agent took. CMS’s WISeR program, which went live in January, has put AI-supported prior authorization directly into traditional Medicare workflows across six pilot states, and the major EHR vendors are pushing out their own agentic features now across revenue cycle, scheduling, and clinical workflow. Most health systems have an architecture that is already strained by read-only AI. When the EHR vendor turns on agentic features by default, and they will, on the vendor’s timeline rather than the CIO’s, the gap is going to be visible inside a quarter.
This is not a theoretical concern. The pattern I expect to play out across health systems in 2027 is not hard to forecast. Pilots that worked in 2025 will start stalling in production. Integration teams will burn out maintaining the point-to-point integrations they accumulated. One or two visible incidents will put the whole AI program under board scrutiny. Programs in this position typically take about two years to put back on solid footing, and the cleanup work consistently consumes a multiple of what the original architecture investment would have cost. I watched something very similar play out, in slow motion, with EHR implementations in the 2010s. The bulk of that cleanup work was preventable.
None of the practical work here is glamorous. Invest in the data and integration layer before the next AI procurement is approved, instead of after. Stand up a curated, FHIR-native data layer that sits between the EHR and the AI tools, and require every new vendor in your procurement pipeline to read from that layer rather than build its own bypass back to the EHR. Move governance, including model versioning, lineage, audit trails, and human review, down to the platform layer rather than letting each application carry its own. Treat agentic capabilities as architecturally separate from generative ones, because they genuinely are. This work doesn’t make for an exciting board slide. That is one reason it keeps getting put off.
The next eighteen months will separate two kinds of health systems. The ones that put the data and integration foundation in place now will see every AI investment that follows build on the last one. The ones that keep buying AI tools on top of an architecture that was never built to carry them are going to spend the back half of the decade doing the foundation work that should have been finished in 2026. Either way, the AI itself was never the hard part of this.
Photo: zhuweiyi49, Getty Images
Vallikranth Ayyagari is a technology leader with more than a decade of experience designing cloud-based platforms, data-integration systems, and interoperability solutions across large healthcare and clinical IT organizations. His work focuses on FHIR-native microservices, real-time healthcare data exchange, and the cloud data architectures that make analytics and agentic AI viable in clinical settings. He is the author of peer-reviewed and industry publications on healthcare data integration, including work on FHIR and cloud APIs for healthcare interoperability, the Model Context Protocol for agentic AI, and FHIR-native architecture in healthcare IT. He writes and speaks on the architectural decisions that determine whether healthcare technology programs scale or stall.
This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.
