Uncovering the Clinical Blind Spot in AI Governance | Executive Interview with Dr. Jen McKay

This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

Uncovering the Clinical Blind Spot in AI Governance | Executive Interview with Dr. Jen McKay

[00:00:03] Hey everyone, [music] it's Google Chrome Friday and today I have questions about AI governance and what good AI governance looks like. This is part of our Google Chrome Friday series for conversations that are running through the month of July. And my guest today is Dr. Jen McKay, the CMIO at Google for Health. Thanks for being here, Dr.
[00:00:24] McKay. It's great to see you on the show. Yeah, great to be here. Thanks for having me. Tell me a little bit about yourself and then I have questions. Okay. Well, I'm a physician by background and before I came to Google about four years ago, I was a CMI at a big health system in the Midwest and I practiced medicine for about 20 20ome years. I'm an internal medicine
[00:00:47] hospitalist and I actually grew up in rural South Dakota and practice medicine in rural South Dakota and I've been lucky enough to keep living here in Sou Falls. So, it's been about four years and I'm pretty much doing the same work that I did when I was a CMO at a health system. And I think the biggest part of my job is translating the humans into for the technical teams to make sure
[00:01:11] that when we put technology out into the world that it actually does what it's supposed to do and it helps people and it helps their job to be easier for them.
[00:01:20] It's amazing. I love people who make that transition from frontline health care. I've done it. I've been in the trenches. I've spent my whole life there. And then they make this transition to partners because it really brings the real world into that partner space. They get to hear from you in real time every day about No, no, no. I know you might think that's what's happening,
[00:01:43] but it's actually this. Well, and truly, my career is really kind of an oddball career. I I would have never guessed at the beginning of my career that I would be working at Google. It's been really awesome. But one of the sort of side journeys I made was I I went to Northwestern University and it was right at the beginning of sort of the patient safety, healthcare quality movement. And I got a degree in healthcare quality and patient safety.
[00:02:09] And the bulk of that really was around human factors engineering, medical error, and really this idea of culture change in healthcare in service of just making it easy to do the right thing.
[00:02:21] And so I got my start really around just medical error and medical error prevention and it really translated very nicely into implementing EHRs and understanding where workflow needed to improve. And I I would say the secret sauce for me has always been just watching the work for real because the work you think is happening isn't really not it most of the time you're actually
[00:02:45] wrong about what you think is happening. So it's true going to the going to where the work is done the Toyota production system kind of mantra of go and see the work and then you'll actually know you won't be getting it secondhand. So let me ask a question given all of that and the question really is about where are health systems getting AI governance wrong right now and what's at stake if they don't get it
[00:03:11] right if they don't course correct. I think that AI governance right now has a tendency to skip over the need for clinical leadership. Um, that's a pretty consistent theory that I've seen and oftent times I'll actually get pulled into projects where there's people are struggling to get things off the ground or to scale and once you add a clinician
[00:03:37] in the mix, it actually unlocks the frustration. I I know that a lot of organizations are, you know, especially physician leadership is really valued it and they're very respectful to that clinical time. However, when you bring clinicians in at the beginning, it really can change how quickly you can go and also how well you do it.
[00:03:59] You're kind of describing the you can do the project really, really well, but you need a clinician to actually pull it into place, right? You do. And it and actually it it's helpful too for things like change management. Change management in any enterprise is really hard. It's especially hard in clinician in the clinical world because disrupting a workflow especially like if it's 2 in the morning and you're on call and
[00:04:23] you're really busy. It can really be the difference between accepting a tool and then and then or maybe not. And my experience in healthcare is that if you don't get it right, losing that trust out of the gates of your clinical community can be really devastating for doing more work, it can take a really long time to like gain that trust back.
[00:04:45] Yeah. So when you think about good AI governance from the clinical side, it's really that it's really about trust and partnership and tell me more.
[00:04:56] Exactly. And I think understanding the workflows a lot of times like I mean the workflows are really clunky. I I always think in healthcare the humans are actually the API for a lot of different systems. So most of us think about EHRs as being this the point of contact and and it's true. EHRs are a huge part of the workflow, but there's the all the other stuff too. And you think about,
[00:05:21] you know, texts and pages and phone calls and, you know, nurses coming up to you at the nurses station, the data inputs you get are numerous. And then to be able to triage what's the most important thing, that becomes the real challenge. And so I think that when you think about just AI in general, you have to remember that I think in the clinical world, the human element of having to
[00:05:47] triage all of the inputs of information is something that isn't really experienced in other industries. I don't think I think the human is the API is a pretty I will be stealing that analogy. I think it's a really good one which you know kind of a take a little side branch here but you know the another question would be a lot of us are using these tools now to educate ourselves on our own health
[00:06:12] issues and those sorts of things. Are we eventually going to replace medical teams, doctors, others with um with AI? I don't think so. I I think you know if you go back to telephone operators back in the day we don't have telephone operators anymore but we have a complete telecommunications industry. I think the work fundamentally changes and I
[00:06:38] certainly think that in my own work using AI the depth of knowledge and the depth of understanding I can gain more quickly.
[00:06:47] It's a little bit more similar to what happened with libraries and card cataloges. So I don't have to go spend 80% of my time writing a paper looking up my sources. Now I can spend much more time synthesizing ideas. So I think that's one component that will change.
[00:07:04] Secondly, I think if we get AI right, it's not just time we're going to give back to clinicians, it's their attention. And the most important part of healthcare, in fact, the best stuff I've ever done in healthcare had a lot to do with my attention.
[00:07:18] Yeah. and that human connection that we all have and that we all need, especially if you're scared and you don't know. And so I think I think some I think the work will change. I think the way that we attain knowledge will change. I certainly think medical education will change and probably in some ways be more more fun um as a result of AI. But I I think the humans the humans are why we have AI in the
[00:07:45] first place. And and I think it will allow us more time for creative problem solving in a way that we haven't had for a long time. It's really I mean it sounds a lot like those who wind up being the best at this or those who can use AI to help them build more trust between themselves and the patient and themselves and their peers which is kind of cool. So, one last question. What would you tell a
[00:08:11] CMIO whose organization's adopting AI sometimes what feels like adopting AI faster than the policy can keep up? Oh. Mhm. Well, I would remind that CMIO that, you know, back when we first started having, you know, cell phones and especially photography, um, uh, there was a lot of things that happened around, you know, like I'm going to snap
[00:08:36] a picture of this X-ray and I'm going to send it to my colleague from the emergency department so that they don't maybe have to come in and they can just look at that. So what I would say is first and foremost people are doing things with AI that make their life easier and the signal that you can that often is the signal that you need to look at efficiency in your processes is where people are using AI and maybe it's
[00:08:58] not being endorsed by the organization. Use that signal to your advantage to say where do we need to sh up processes that maybe aren't as helpful. The thing second thing I'd say is in that same vein that also opposes security risk.
[00:09:12] And so when it comes to governance, really be thinking about and partnering with your security team, your compliance team because we've all been in had experiences in our in our careers where you weren't thinking about PHI being a problem and and maybe it was. And then the last thing I'd say is that really think about AI as an efficiency driver
[00:09:38] and a workforce multiplier. Using AI for the sake of using AI is not the right way to go. And where I've actually found the mo most success in the work that I've done is when you're really using it to solve an actual problem. You know, a lot of times what we'll see, especially with physicians, is they'll go to a conference and they'll see this awesome like point solution, but truly AI is actually the most effective in those high volume,
[00:10:03] high frequency, high errorrone workflows. Documentation is a really good example and I think that's why we're seeing such rapid adoption in the ambient listening world. So that that's probably be the most important thing.
[00:10:15] And then the other thing I would say is that one of the big and this is maybe something maybe not so focused on clinical but because we live in a world where EHRs are really the primary sort of data input transactional systems that people use that also lends itself to an IT group or an IT division that focuses on upgrades maybe once every couple of years. And the organizations I'm seeing
[00:10:42] really take off with IT are ones where they've had a corresponding growth and upskilling of their IT workforce to the extent that they behave or perform more like product managers within a big tech company might perform. So that that's an area that I think is probably more interesting to CIOS is how do we get this rapid cycle innovation going throughout our organization that not just supports the work that's already
[00:11:06] happening but also potentially uses AI to sort of surround the ecosystem because EHRs are not going to go away. They're part of the part of the ecosystem. They're part of the world.
[00:11:18] But it's the everything else tooling that you can really bring value to your your human API or some as we sometimes refer to them as physicians. Wow. I appreciate it. Really great advice and I will really appreciate you being on today. It's fun to spend some time with you, Dr. Jen McKay. I hope you I hope to see you in person on the road somewhere soon.
[00:11:39] That that'd be awesome. It's It's always fun to talk about this stuff. It's moving fast and furious and I feel really really lucky to be working at Google at this time and I get to meet all kinds of fun people and do all kinds of fun projects. So, whatever I can do to help other people be successful, I'm more than happy to have that conversation.
[00:11:58] Thanks for joining this executive interview with me, Drex Deford. Here at This Week Health, we believe every healthc care leader needs a community to lean on and learn from. Build your network at thisweek.com/subscribe and share all of this with a colleague.
[00:12:14] Thanks for being here. I'll see you around campus. [music]