AI governance is becoming a normal part of running a health system. That makes sense. UPMC and KLAS recently reported that more than 90% of health systems surveyed had deployed third-party AI solutions. Clinical documentation was the most common use case, reported by 52%, followed by revenue cycle, coding and billing at 36%.
Once AI starts showing up in real workflows, the questions get serious pretty quickly. Which tools are approved? What data can they access? How were they tested? Who is responsible for reviewing the output? What happens when the system gets something wrong?
Most of this work, though, is focused on AI that the health system has chosen to use.
There is another side to it.
Patients are already using AI to ask questions about their health, find physicians, understand treatment options and decide whether they need care in the first place. KFF found that 32% of U.S. adults had used AI for health information or advice in the previous year. Among those users, 41% said they had used it before deciding whether to see a healthcare provider.
That 41% is the number I keep coming back to.
It means these tools aren’t being used only to explain what an MRI result means or define an unfamiliar medical term. For some people, they are becoming part of the decision about whether to seek care and, potentially, where to seek it.
Health systems have very little control over that conversation.
So while we’re getting better at answering “How should we govern the AI we use?”, we probably need to start asking another question as well:
What about the AI our patients use?
Your website can be right and the answer can still be wrong
Take a fairly normal example.
Someone is looking for a neurologist who treats migraines. They ask an AI assistant and get a recommendation for a physician at your health system.
Great.
Except that physician left six months ago.
Or perhaps the physician is correct but the location isn’t. Maybe the answer points to an old profile. Maybe your organization doesn’t appear at all, even though you have a major neurology program in that market.
This is where things get awkward from a measurement perspective.
Nothing necessarily breaks on your side.
The website may be correct. Analytics are working. The CRM is working. Your physician directory has the right information.
There may be no visit to record because the patient never reaches you.
From the health system’s point of view, nothing happened. From the patient’s point of view, they may already have crossed you off the list.
We’ve spent years getting better at understanding what happens once someone enters our digital ecosystem. We have much less visibility into what happens immediately before that.
The source-of-truth problem has changed
Anyone working in healthcare marketing knows how difficult it is to keep information current.
Physicians move between locations. Hours change. Service offerings change. Insurance information changes. New locations open. Old profiles survive online for years.
Health systems have put a lot of effort into cleaning this up on the properties they control. Better physician directories, better location pages, structured data, more consistent service-line content.
That work still matters. Probably more than ever.
But having a clean website doesn’t mean the rest of the internet is clean.
A recent study looked at AI Overviews for 100 brands and analyzed 652 sources cited in the answers. Only 29% of those citations pointed back to the brands’ own websites. The other 71% came from somewhere else.
This wasn’t a healthcare study, so I wouldn’t apply the 71% directly to hospitals. But the finding is still useful because the underlying problem is very familiar.
Your website is only one version of your organization available online.
There are directories, news stories, review sites, government databases, partner websites, physician profiles and old PDFs. Some are current. Others aren’t. Sometimes two credible sources say completely different things.
An AI system has to make sense of all of that.
For years we’ve asked, “Is the information on our website correct?”
Now we also need to know whether that’s the information AI is actually picking up.
Even branded search isn’t as simple as it used to be
This isn’t limited to people asking broad questions like “best cancer hospital near me.”
An Ahrefs analysis from September found Google’s AI Overviews appearing in 82.91% of U.S. desktop results for branded keywords on the date they measured.
Think about what that means for a hospital.
Someone already knows your name. They search for you specifically. Traditionally, that’s about as straightforward a digital journey as you can get.
Increasingly, there can be an AI-generated layer sitting between the search and your website.
So I don’t think we should treat this purely as a patient-acquisition issue. It also affects people who already know who you are.
The question isn’t just whether your hospital ranks for its own name. Of course it does. The more interesting question is what gets presented around that result, where that information came from and whether it matches what you would want a patient to know.
“How often do we show up?” isn’t enough
A lot of the AI visibility conversation has naturally started with rankings and share of voice.
I understand why. They’re relatively easy concepts to explain.
If Hospital A appears in 70% of tracked answers and Hospital B appears in 45%, Hospital A seems to be doing better.
But add one more piece of information.
Hospital A appears more often, but some of those answers contain outdated physicians, incorrect locations or confusing information about services. Hospital B appears less often, but its information is consistently accurate.
Now the 70% doesn’t tell us nearly as much.
For healthcare, I think we need to look at three things together: how often the organization appears, whether the information is correct when it does appear, and how those answers change over time.
That last one is easy to overlook.
AI answers aren’t static pages. A result you check today isn’t necessarily the result a patient gets next week.
One recent brand-search study repeated 44 identical queries. Sixteen of them produced a different result when they were run again.
That’s more than one in three.
Again, not healthcare-specific. But it makes the problem with one-time audits pretty obvious.
Checking ChatGPT on Friday afternoon isn’t a monitoring strategy
This is probably how most organizations start, and there’s nothing wrong with it.
Someone on the marketing team gets curious. They type in the hospital name. Then a service line. Then a few competitors. Someone creates a spreadsheet and starts copying answers into it.
You can learn a surprising amount that way.
The problem comes later.
Let’s say you discover that an important cardiology service isn’t being represented correctly. The team updates the service page. Someone improves the structured information. An outdated third-party profile gets corrected.
Two weeks later somebody asks: did it work?
That’s a harder question.
You know the website changed. You know the profile was corrected. Maybe organic traffic moved.
But did the answer change?
And if it did, was the change temporary or did it stick?
This is the point where occasional checking stops being enough.
So what is actually worth watching?
I wouldn’t start by trying to monitor every question anyone could possibly ask about a health system. That would create a lot of data and not necessarily much insight.
Start with the parts of the business where the answer matters.
If orthopedics is a strategic service line, start there. What do people ask when they’re looking for orthopedic care in your market? Which organizations are recommended? Which physicians appear? What happens when the question becomes more specific?
Do the same for priority physicians and locations.
Location questions are especially interesting because they combine several types of information. A location can exist, but the AI can misunderstand which services are offered there. A physician can be correctly associated with the system but connected to the wrong office. Hours can be stale.
Then look at competitive questions. Patients don’t always search for one organization at a time. They ask which hospital is better for a procedure, who specializes in a condition or where they should go.
And pay particular attention to questions that suggest the person may actually be close to taking action.
“What’s an ACL tear?” is worth understanding.
“Where can I get my knee checked this weekend?” is probably worth understanding more.
That difference should influence what gets monitored and what gets fixed first.
Not every bad answer deserves a meeting
Once teams start looking, they will find problems.
That’s almost guaranteed.
The mistake would be treating every problem as equally important.
If you find an awkward description of a service in one answer, that’s probably not an emergency. If an AI system repeatedly recommends a physician who left the organization, that’s different.
We use a fairly simple way of thinking about this: how important is the patient question, how serious is the problem, and does it keep happening?
You don’t necessarily need to turn that into a complicated score.
It’s mostly a way to stop a team from spending three days chasing an odd answer to a low-value question while ignoring something that could genuinely confuse patients.
This is also where healthcare-specific context matters. “Wrong” can mean very different things depending on the information involved.
Before changing the website, find out what might actually be wrong
Another temptation is to find an inaccurate answer and immediately rewrite the corresponding page.
Sometimes that’s exactly what needs to happen.
Sometimes it isn’t.
If the website already contains the correct information, changing it again may do very little.
Look around first.
Is there an old physician profile somewhere?
Does a major directory have the wrong location?
Are two pages on your own site saying slightly different things?
Is an old PDF still indexed?
Is another authoritative source using outdated information?
The useful question isn’t “How do we manipulate this answer?”
It’s “Why might the system have reached this conclusion?”
That’s a much better starting point, especially in healthcare.
Then somebody has to own it
This is the part I expect a lot of health systems will have to work out over the next couple of years.
Who owns an inaccurate AI answer?
Marketing cares about visibility. Digital owns much of the website. Provider-data teams may own physician information. Patient access knows what is actually bookable. Communications cares about reputation. IT and governance teams are increasingly responsible for AI policy.
An AI answer can touch all of them at once.
I don’t think the solution is creating another giant committee.
A simpler operating model is probably enough.
Someone — most likely marketing or digital — needs to own the monitoring. Their job isn’t to personally fix every problem. Their job is to know what’s happening and flag what matters.
The team that owns the underlying information should validate whether something is actually wrong.
Then marketing or digital can investigate where the discrepancy may be coming from and coordinate the fix.
The governance piece is deciding what deserves escalation. A missing brand mention and incorrect information that could send a patient to the wrong place obviously aren’t the same thing.
That distinction should be made before there is a problem, not during one.
If you’ve never looked at this, spend 30 days on it
You don’t need a six-month AI strategy project to find out whether this matters to your organization.
Pick one service line.
Not the whole health system. One area that matters commercially or strategically.
Then write down 20 or 30 questions a patient might genuinely ask while looking for that type of care. Mix broad discovery questions with physician, location and comparison questions. Include a few where the person sounds close to making a decision.
Run them and record what you find.
Don’t obsess over building a perfect score. Look for patterns.
Are you consistently present?
Does one competitor appear much more often?
Are the same physicians showing up?
Are there obvious errors?
Do the answers change when the question becomes more specific?
What sources keep appearing around those answers?
Then choose the three or four findings that actually matter and investigate them.
Correct what you can.
And repeat the exercise.
The most important question at the end of the month isn’t whether your “AI visibility score” went up.
It’s whether you can now answer these questions with some confidence:
What is AI saying about us?
Where are the gaps that actually matter?
Do we understand what might be causing them?
When we make a change, can we tell whether it worked?
If the answer to the last question is no, you have a measurement problem.
This is what we mean by AI Search Observability
We’ve been using the term AI Search Observability at Litode because “AI visibility” doesn’t quite describe the problem anymore.
Visibility is part of it. But knowing that your hospital appeared in an answer doesn’t tell you whether the answer was accurate. It doesn’t tell you why a competitor appeared instead. And it certainly doesn’t tell you whether something changed after your team took action.
Observability is closer to what healthcare teams actually need: enough information to understand what is happening, notice meaningful changes and investigate when something looks wrong.
The basic loop isn’t complicated.
See what is happening. Keep watching it. Investigate what matters. Make a change. Check what happened afterward.
The hard part is doing that consistently across a health system with hundreds or thousands of physicians, multiple locations and service lines, and information spread across properties the organization does and doesn’t control.
There are now two sides to healthcare AI governance
The numbers tell an interesting story.
More than 90% of health systems are already using third-party AI.
At the same time, 32% of U.S. adults have used AI for health information. Among those users, 41% say they have used it before deciding whether to see a healthcare provider.
Health systems are adopting AI from one direction. Patients are adopting it from the other.
Most of the governance conversation has understandably been about the first.
I think HCIC conversations this year are a good moment to spend more time on the second.
Not because every inaccurate AI answer needs to become a compliance event. And not because healthcare marketers need another dashboard full of numbers.
The reason is simpler.
If these systems are becoming part of how patients understand hospitals, physicians and care options, someone inside the organization should have a reasonably good idea of what they’re saying.
You can’t fix every answer. You can’t control every source. And you definitely can’t control the models themselves.
But you can know more than you know today.
For me, that’s where the next part of the AI governance conversation starts.
Not with another policy.
With a fairly simple question:
If AI changed what it was saying about your health system tomorrow, would anyone on your team know?