Healthcare Isn't Ready for an AI Workforce Yet | Newsday with 229Project

This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

Healthcare Isn't Ready for an AI Workforce Yet

[00:00:00] Bill: I'm concerned that we are only at level 2 in healthcare, and we need to push past that pretty quickly, and I'm looking for 27 to push

[00:00:10] Bill: into more AI workforces and AI agents.

[00:00:19] Bill: Alright, it's Newsday, and it's the three of us. The three of us are back.

[00:00:23] Sarah: Wow.

[00:00:24] Bill: awesome. And, I guess, Drex DeFord, Sarah Richardson, myself, three former CIOs, I guess, now's as good a time as any to say, we will have new people joining us. We have, Lori Boisjoli, who is, a former CIO, University of Vermont.

[00:00:43] Bill: a lot of, development background and, really good, resilience background as well. So, we are, really excited to have her join us. And we also have,

[00:00:56] Bill: Andrew Rosenberg. I think Lori will be on this show. Andrew, I'm not sure yet. He's, still work… he's still doing some other stuff, but he's gonna join us and do the, 229 Fellowship, and from time to time, he will be showing up on the show as well, so we will have to… we will have to up our game, I guess. Or they'll have to up their game, I don't know. I don't know which it is, I don't know.

[00:01:16] Drex: Whatever we do, we gotta make room on the screen, that's all I know.

[00:01:18] Bill: Right. I don't know how that works.

[00:01:21] Drex: auto… I think it auto-adds principals when the time is right.

[00:01:25] Bill: So, so we should be good. You, we've all been traveling, but you guys, Kansas City and LA Regional Staff Roundtables, I went to the conversation in San Diego.

[00:01:37] Bill: Give me your update from… from those two meetings. What are you hearing?

[00:01:42] Drex: And Kansas City was, fantastic. It was a smaller group.

[00:01:47] Drex: but obviously a tight-knit group. One of the things we hear everywhere we go is the… it was really nice to see people that I hadn't seen for a long time, like, to renew acquaintances and to make them stronger and more of a tight relationship.

[00:02:02] Drex: And it was nice to meet a bunch of new people that I had never met before, but I know that they're in the market, I've just never had the chance to meet them. So, that connection wound up being…

[00:02:12] Drex: Really great. In both markets, really, but in Kansas City, it just seemed like, they were, they were really, really into it. Great conversations.

[00:02:23] Drex: Deep discussions on both AI and resilience.

[00:02:26] Drex: And, yeah, I mean, folks stayed, late… later into the day, which was kind of cool in both cases.

[00:02:36] Bill: That's awesome. Sarah, tell me about the, so AI and resiliency, what did you hear?

[00:02:43] Sarah: Here's what I find so fascinating about our resiliency conversations, and we have them all across the country, is literally, like, markets.

[00:02:52] Sarah: that have had an event, whether that's crisis management from weather, or it's been a major outage, or it's been an outage that has then created, like, a diversion to another location, clearly they have the most

[00:03:06] Sarah: to share, and the perspectives on how to really be prepared for something that might be happening, I love that aspect of it, because we've been to markets where nothing significant's happened, and I worry for them, because they don't know what to do if something happens, because they haven't had that sharing. Because in markets, let's be honest.

[00:03:22] Sarah: Most people have worked at 2 or 3 of the health systems inside a given market, regardless of the size of the city. We find that quite a bit, so they've been able to share those experiences.

[00:03:31] Sarah: But what's most interesting is when we dive into that resiliency conversation, is that

[00:03:37] Sarah: nobody is yet prepared for the AI resiliency that's required, because it's new technology and it's moving fast. So I anticipate next year, the maturation of AI stays part of the conversation. AI resiliency, to a degree, becomes part of the morning conversation. We'll have a different topic in the afternoon, but I'm continuously leaning into

[00:03:56] Sarah: AI resiliency, and it's a lot of what Drex touches on in the 2-minute drill, is

[00:04:00] Sarah: How do you prepare for a bot swarm that takes over, or just some of the unknown things from a third-party risk perspective that are creeping in? So, we're definitely on to something. I'm glad we created a safe space for them to talk about it amongst their peers.

[00:04:38] Bill: AI Swarm. Drex, are you telling…

[00:04:40] Bill: Stories, again, that are scaring the kids.

[00:04:43] Drex: I'm not… I'm not, actually. Just maybe, maybe some… it is Halloween, and if you, saw the header for the Two Minute Drill Extra, it was a ghost story, with a light flashlight under my chin, not my chin, yeah.

[00:04:57] Sarah: fun.

[00:04:58] Drex: But yeah, there's some scary stories out there. The other thing about the AI resilience part of this, Sarah, is the reality that many of them

[00:05:08] Drex: have in the, you know, take, shape, and make model. Most of them still take the AI from their partners, so they're relying on the partners to have… help them through that resilience conversation.

[00:05:19] Drex: The shape part, same thing, they're going to rely on the partners, but the make part, the folks who are starting to build AI themselves, or realize they have a lot of shadow AI in the health system.

[00:05:31] Drex: That's the part that has a lot of them worried. I, you know, it's new to them. We've not been builders for a long time, and so this whole new piece of work that a lot of folks have taken on, or that they think their departments have taken on, maybe behind their back, has them worried from a resilience perspective.

[00:05:49] Sarah: Yeah, they're only true…

[00:05:51] Sarah: chat, or the ability to utilize it most effectively, is Copilot, because it's built into their Microsoft.

[00:05:56] Drex: A lot of them are focused on co-pilot.

[00:05:58] Sarah: Most of them co-pilot as their co-pilot, and not the level of, like.

[00:06:04] Sarah: I think dissemination between… I'm going to stratify different levels of problem solving with different AI agents, which is what we get to play.

[00:06:10] Bill: Copilot is no pilot. You know what that is? Copilot is co-pilot. I don't…

[00:06:14] Sarah: Yeah, how are we gonna introduce that effectively? Like, and Bill, you do it quite a bit here, but, like, how do you get to the space where

[00:06:22] Sarah: the ability for these CIOs to have these mini agent factories that are not sponsored by their EHRs, part of the sandbox that they are continuously playing in is something that's top of mind, and definitely what I believe, they will seek our assistance in creating, to a degree.

[00:06:40] Drex: All these things are connected together, too, right? A lot of it's about resources, and Copilot is free, and so that's what they use, and that's what they leverage as… that's what they leverage, and that's what they focus on.

[00:06:51] Sarah: Puppies are free too, Drex.

[00:06:53] Drex: Yeah, puppies are fun.

[00:07:29] Bill: Five levels of AI usage. We talk about this from time to time. So the first was Google replacement, right? I ask it a question, it gives me an answer. Second is Copilot, help me to do a, you know, a task. You give it a task, and it gives you something back. Not Copilot.

[00:07:43] Drex: Pilot the product, but as a co-pilot.

[00:07:46] Bill: as a co-pilot, and quite frankly, most people are using Copilot as level 1 or level 2.

[00:07:52] Drex: Right.

[00:07:52] Bill: Level 3 is where you're an agent. This is where you're giving it a job, and it works for, like, 2-3 hours, could work through the night, and it comes back, and it gives you, some things. And…

[00:08:03] Bill: I'm not finding many health systems are hanging out in that space. It's mostly Level 1, Level 2.

[00:08:08] Bill: we, as a company, are leaving Level 3, and we're moving to level 4, and so Level 3 was give it a job, and I was playing around with this, and people were reading my stuff, I was playing around with this for a while, and we were… and I was getting Claude and other things to, like, program through the night, and do really cool things.

[00:08:27] Bill: It's this loop programming and graph programming and that kind of stuff. It's complicated, it's complex. It wasn't easy to roll out to the entire organization.

[00:08:37] Bill: And on… and 2 weeks ago, we started rolling out these bots. Now, a bunch of them have them. Claude just came out with theirs.

[00:08:44] Bill: OpenAI has dots, Grok has GrokBot, and Meta has Muse.

[00:08:50] Bill: Right? And this is where you set up an AI workforce, and it's so simple. Like, you see all these videos, because these people are setting them up, and they're like, oh my gosh. So all I did for some of our staff was I just gave them a license.

[00:09:05] Bill: And you were on a call, you probably didn't even notice this, you were on a call today, and Jessica was going through, and she was… she had her 4 bots there. Now, she's on 4 bots, I have, like, 12, and one of them's our CFO, one of them… I mean, I have 3 dev bots, I have a CFO, I have a whole bunch of things, and it's unbelievable what the work it's doing. That's a level 4, by the way.

[00:09:28] Bill: Level 5 is an AI-native organization. We can't even talk about that yet, because we're nowhere near it, but it's interesting to have the AI workforce, because it's changing the way I'm thinking. I think there's going to be AI at the workflow, but then I think there's going to be

[00:09:42] Bill: this whole AI workforce that we are… Drex is gonna have his, you're gonna have yours, Sarah, I'm gonna have mine, and it's gonna do work in the system. I'll give you an example. Jairus

[00:09:53] Bill: Gave him access, and he started setting up his bots.

[00:09:56] Bill: And, I said, well, man, I haven't turned on the API for you, you can't get into HeLOS, which is our backend. He goes, oh, it's in HeLOS. I'm like, it's in HeLOS?

[00:10:07] Bill: He goes, oh yeah, it has its own computer, and it said, oh, do you want me to log in? So, I went over, and Drex is gonna say, don't do this, but he gave it his credentials, it logged in.

[00:10:16] Bill: And so now, anytime you ask the questions about partners, or events, or whatever, it goes over, it takes… it goes into that thing. Now, by the way, this is a very inefficient use of tokens.

[00:10:27] Bill: a better use of tokens would be to give them the API, and we will, we'll correct that, but, because it's just a lot quicker for it to just do database calls and get the information back. But right now, it's just doing that whole thing of going through, and it's giving information, and I saw… I was with him on Thursday.

[00:10:46] Bill: And, he told me the job he got done, he goes, Bill, this would have taken me 4 hours, and it took me, like… I gave it the job, we went and had lunch, we did something else, he came back and the job was done. Like, you don't even have to… somebody else is doing the work.

[00:10:59] Bill: For you, almost. I am concerned, and so, I'm getting close to the end of our regional staff roundtables. I have Boston this week, I have, Charlottesville. Right.

[00:11:11] Bill: I'm concerned that we are only at level 2 in healthcare, and we need to push past that pretty quickly, and I'm looking for 27 to push

[00:11:21] Bill: into more AI workforces and AI agents. We had, Eric Larsen spoke at the conversation. This is a Bridges event they do once a year. It's a very different event. It's,

[00:11:34] Bill: For example, I think they gave… they talked about a bridge.

[00:11:38] Bill: Shiv talked about a bridge for maybe 10 minutes.

[00:11:41] Drex: Yeah.

[00:11:42] Bill: That was it. And you know what he was talking about? This whole concept of your agent within the EHR, they've… they've layered something on top, and it's really interesting.

[00:11:51] Bill: But the.

[00:11:52] Drex: Yeah, go ahead.

[00:11:54] Bill: The rest of the event is about creativity and humanity and overcoming, and I got to meet Alex Smith, and I just want to say that again in front of Sarah, say I got to meet Alex Smith.

[00:12:05] Drex: I was like, who's that? Who?

[00:12:06] Sarah: Did you get me anything signed?

[00:12:08] Bill: I didn't get anything. Well, he signed my foot.

[00:12:11] Sarah: hard hat, I could have given you soap.

[00:12:13] Bill: Oh, my God.

[00:12:14] Sarah: For my office to have him sign, just for the record.

[00:12:17] Bill: If you haven't had a chance, if you, Google Alex Smith's leg and look it up on the internet, it is the most grotesque thing you've ever seen.

[00:12:25] Sarah: documentary he did, I'm on it, too.

[00:12:27] Bill: And he tells the whole story, it is a… it is a beautiful story of resilience and, making a comeback. I didn't realize he played that many years.

[00:12:36] Bill: In the NFL. He was… he, actually, as far as walking for NFL quarterbacks, he looks pretty good. Have you seen… some of the NFL quarterbacks, when you see him years later, it's like, man, wow, must be hard to get out of bed. By the way, congratulations, Drex. Nice game last night.

[00:12:53] Drex: It was a… it was a good game, kind of fun and interesting, weird that, you know, they… that's not the team that shows up every week, unfortunately, but yeah, they played great last night.

[00:13:04] Bill: That is… that is tricky.

[00:13:06] Sarah: It was a fun football day in this household. Hey, how come you don't give your bots their own identities and login information yet?

[00:14:02] Bill: Oh, no, I do. Well, that's who you're… you're talking to Albert. Albert has his own email, has his own login, has his own… all that stuff.

[00:14:08] Sarah: Right. Well, there

[00:14:09] Sarah: you'd said that, like, if Jairus is giving it his credentials, I'm like, this becomes how we also foster the knowledge.

[00:14:15] Bill: Yeah, this is…

[00:14:15] Sarah: HR…

[00:14:16] Bill: That's what I've been writing about, which is, I don't think you can stop it.

[00:14:21] Drex: There's the, there's actually…

[00:14:22] Bill: on a browser, I've looked for, like, the signature of an agent logging into our system.

[00:14:30] Bill: I can't deter… it's logging in on Chrome, it's… I mean, I can't do it by IP address, because you're always moving around. I can't do it by… I mean, I guess I could do it by MAC address. I don't… I mean, but that's really specific.

[00:14:42] Drex: There, there must be a way to do it.

[00:14:44] Drex: I'm sure there is.

[00:14:46] Bill: Well…

[00:14:46] Drex: And here's why I know. Because if you read Two Minute Drill Extra this week, I've referenced you, and the story, and the Don't Share Your Password, but the banks, all the big banks now have…

[00:14:58] Drex: Take an action to… cut off access by AI

[00:15:04] Drex: assistance, because it's creating a lot of problems inside the banking system now. Transferring money, doing things that is not intended that the AI do, so they figured out a way to sort of cut those… cut those bots off the past.

[00:15:21] Bill: Only I had a billion-dollar budget to do this. I know. I have heard, because I talked about this somewhere, and CrowdStrike's agent can identify some of this stuff, and so can… but they identify it at the workstation.

[00:15:38] Bill: Well, we don't… we don't… and Paolo and some others, but we… we can't… anyway. It'll be interesting to see. I think everybody's gonna have to be able to figure out if it's an agent coming in, or it's not an agent coming in.

[00:15:51] Bill: To the system. Very different usage patterns, talked about this, and Drex, you commented on, hey, don't give away your password, and my comment back to you was, you know, we had this argument before with cloud, and people were like, hey, stop using the cloud as a CIO, and the people would just look at me like, and use what?

[00:16:11] Bill: use… and I'd be like, you're right, we have to go.

[00:16:14] Drex: I mean, look, there's a lot of things that come out of…

[00:16:17] Drex: our conversations with our end users, and they get upset because their solution to the problem is to share their password with their agent. And we know that's not a good answer.

[00:16:29] Drex: So, no, don't do that, can't be the final answer. What we're getting from them is a signal that they need to solve a problem, and this is the only way they have to solve it right now, and we need to help them find a better way to do this.

[00:16:44] Drex: In everything that we do, create a really easy way for them to do the right thing, and create a really difficult way for them to do the wrong thing. And if we can do that, we're gonna all be better off.

[00:16:55] Drex: But yeah, don't share your password. Let's just start there, and understand that we've got to figure out a better way to make that work, because that's not a good solution.

[00:17:05] Bill: We… we are all in agreement. We are all in agreement. And thank you for calling me out on that, Sarah. Drex Two Systems took the… took themselves offline in one week. Parkview Health.

[00:17:16] Bill: University Hospitals Cleveland cut its internet connection after unusual activity on October 22nd. It began rescheduling some procedures, and on Friday, it said the impact would last into next week. Neither has said whether it was an attack.

[00:17:30] Bill: What does that look like? I mean, what kind of activity is going on that people go, hey, that doesn't look right, let's shut it off? I mean, and when they say shut it off, what are they doing? Like, the little plug to the internet, they unplug it?

[00:17:42] Drex: Yeah, it could very well be something like that. It may be that they're taking individual systems offline, or they're taking the front end of their…

[00:17:50] Drex: patient interaction, their portals, they're taking those offline, and clearly that's happening based on some of the information in the stories, but it could be attack, it could be a lot of things. This also could be an agent that's out of control inside the environment. It could be any number of things, and the sort of

[00:18:09] Drex: typical response in all of these things is to not say anything. We're having a…

[00:18:15] Drex: a computer problem, and we're gonna be offline for a while, and we'll let you know when we're back up. So, this is the Lawyer-up mentality that we have as soon as something bad happens, and it becomes clear to patients or others that they can't get to the system anymore, the health system anymore.

[00:18:33] Drex: the response is usually, we're working on it. And, that lack of transparency is what drives a lot of rumors, and a lot of other conversations. It wastes a lot of time sometimes, so I wish we were more transparent, it's just the way the… it's just the way the world is built right now.

[00:18:51] Sarah: I keep wondering of how much of the agent activity, and we've already seen it, with the usage going high in organizations that are allowing some of this agent activity, it's a different type of denial of service, where you have so much activity that it has to be researched, and the safest mechanism is to say, hey, let's close the front door for a few minutes while we validate, but I think these spikes in usage, or these

[00:19:15] Sarah: Spikes are going to either be things like our historical denial of service, or the amount of agent activity

[00:19:22] Sarah: being greater than what the bandwidth can handle appropriately, and so how do we prepare for that effectively, too? If we know that already more than 50% of the activity out there on the internet is bot-driven, how do you have enough bandwidth for it to still meet the needs of… every patient now has an agent or a bot.

[00:19:39] Sarah: working on their behalf. Does that increase usage to a degree that's not as sustainable until we figure out what the balance is going to look like in the middle?

[00:19:49] Bill: It's been… I think the AI conversation has been a little concerning for me with health systems. The resilience conversation has been really encouraging to me, and I think

[00:20:02] Bill: I'm hearing people have plans, people are thinking through this, that it's a lot more mature than it was two years ago, and I'm really encouraged that, first of all.

[00:20:12] Bill: That we're identifying, but second of all, identifying it is not a small deal, but the second thing is having the courage to take it offline.

[00:20:21] Bill: there was a time where we'd be like, no, no, let's just wait to see what happens. Like, you don't want to take it offline, and it's interesting.

[00:20:28] Drex: seeing the conversations that happen inside of an organization, too, once you take it offline. The…

[00:20:34] Drex: evolving maturity of the conversations where there are folks inside the healthcare organization who start to put a lot of pressure on, we need to go back online. That used to be everyone. Hurry up, we need to get back online, we need to connect to this, we need to connect to that. I know that third party got breached.

[00:20:54] Drex: But we really need to connect to them, because it's part of our XYZ system. We need to reconnect to them. Well, they don't… they haven't certified that they're clean yet. It's okay, look, I've talked to the people there, we need to reconnect to them. That…

[00:21:07] Drex: Conversation has evolved over time, and there's…

[00:21:11] Drex: the pressure for that has lowered compared to a year ago or two years ago. People… those inside the health system very much now understand more clearly that they need to

[00:21:27] Drex: take their time before they reconnect to the internet, or they reconnect to a third party, and why that's important. But it's taken a lot of work, a lot of conversations.

[00:21:35] Bill: Sarah, I want to come to you with the next one, because you were a CIO who rolled across the country and assimilated organizations in one of your roles.

[00:21:47] Bill: You know, again, Eric Larsen, the gentleman who spoke, is a former advisory board person, and now with Towerbrook Ventures, or Towerbrook Advisors, or whatever. He speaks to a lot of CEOs, he's written two.

[00:22:02] Bill: Amazing articles on AI, healthcare Oppenheimer moment, the industrialization of intelligence and the future of U.S. healthcare, and the Gen AI juggernaut, the U.S. healthcare is not prepared

[00:22:13] Bill: 4 and,

[00:22:15] Bill: He made a note in his keynote that, we are getting ready for an acquisition boom.

[00:22:21] Bill: And we have two of them in this week's, stories. We have Integris.

[00:22:26] Bill: signed with Risant, and, which is also a part of that, that, that big, Kaiser deal, and so you have Geisinger, Cone, and now Integris, and then you have, Hackensack Meridian Buys Hunterdon.

[00:22:42] Bill: We had Rush a couple weeks ago. What… what could… what could CIOs be preparing for as… if they… if they feel like they're going to be acquired, or an acquirer, how do… how do they get in front of this? What does it… what does it look like to be prepared for this kind of activity?

[00:23:02] Sarah: I would say, first of all, I thought you were gonna say Eric Larsen, the guy who wrote all the, like, historical fiction books that are, like, really cool. Oh, different Eric Larsen. And I think the most… the…

[00:23:11] Sarah: The biggest piece is to take all of your personal fears, uncertainty, doubt, all of those aspects, just toss them out and worry about everybody else, because whatever's going to happen to you is going to happen to you. So rather than go into the protect my job mode, it's the, how do we figure out how to integrate this most effectively? And the pieces that get missed in that due diligence and some of those transition services agreements are the speed by which integration is key.

[00:23:35] Sarah: And I'll always be a fan of the second you know that it's going to be a done deal, integrate it as fast as possible, and be very specific about who needs to align to get the work done, what systems can be brought together and go away.

[00:23:48] Sarah: patch the vendor concerns, patch the security holes, like, anywhere that there is fuzziness, for lack of a better term, address that right away, because some of those peripheral things that you believe you can wait until later end up becoming some of your biggest liabilities, and that is more true now than ever. So, going after it with the intentionality, how quickly can we effectively bring it together starts with the culture.

[00:24:11] Sarah: But the technology is probably commensurate in terms of the importance of making sure you know what you're getting yourself into, on both fronts.

[00:24:20] Drex: Yeah, I would…

[00:24:20] Bill: like this.

[00:24:21] Drex: back to the earlier part of that, even the, like, being involved in the due diligence. A lot of health systems, even today that I talk to, find out that they're acquiring someone, or that they're being acquired, but they find out kind of at the last minute. It's basically a done deal.

[00:24:38] Drex: And they then look at the due diligence that's been done, and they go, wow, we didn't ask a lot about tech debt, we didn't ask a lot about their security posture, we didn't ask about a lot of things.

[00:24:50] Drex: That now that the deal is done, it turns out not to be nearly as valuable as a deal as those who made the deal thought, because there's all these other built-in expenses that have to be… have to be covered.

[00:25:04] Sarah: Yeah, we literally bought something at a previous lifetime, and we found out, IT found out, sort of after the fact, and had to make it work, and after we did a deeper due diligence, we said, this is an absolute train wreck, we never would have recommended it. Had to do it anyway, and guess what we dismantled two years later?

[00:25:21] Sarah: That exact organization that we said you shouldn't have bought it in the first place.

[00:25:25] Sarah: But IT wasn't asked at the forefront because… for myriad reasons, and it wasn't a, hey, I told you so, it was, like, that was a really messy, expensive, impact to both… to all… everyone involved.

[00:25:38] Bill: Yeah, I… this is a, playbooks, playbooks, playbooks, is what I say to people on this. And,

[00:25:46] Bill: you know, if you're doing one for the first time, and you're, like, just figuring out how to do it through it, document the crap out of it, and when you're done, write it up, and then look at it and say, man, if we had to do this over again, what would we do? Organizations that have playbooks do this really well.

[00:26:02] Drex: Talk to your friends, too, right? I mean, you went to the summit with a bunch of people who have probably done acquisitions, send them a note and say, can I see your playbook?

[00:26:12] Drex: Can you share your playbook with me? The other part of the acquisition process is that we still see organizations taking two approaches to this, and we've talked about this in summits and RSRs, too.

[00:26:23] Drex: One of the approaches is, every time we acquire somebody, or we divest somebody, it's a project in and of itself. And the other version of that is, we know that we're going to be in this business all the time, every year. We're just going to build a part of our department that does acquisitions and divestitures. And those people just know what they're doing, and that's how we're going to run our business now, because it's just a constant.

[00:26:48] Drex: So, think about that as part of this discussion, too.

[00:26:53] Bill: Yeah, I mean, whenever we talk about this, I'm like, everything happens at that negotiating table. If it's done right, it tends to go well, and it's all expectations. Expectations are the mother of all disappointment.

[00:27:04] Bill: The organizations that do it well sit down and essentially set the expectation. Your IT systems will be going away, you will be coming onto our IT systems.

[00:27:13] Bill: And when they say, oh, we got some good stuff, it's like, that's great, I'm glad you do. But the cost of running two systems is too high, so as soon as we possibly can, and most of these systems are struggling, right? So they haven't invested…

[00:27:25] Bill: And when you're at that table, that's the perfect time to say, look, we like our systems. They may not be great, they may not be perfect, but we know how to run them, and we have efficiencies, and part of this is bringing you online. Let's see, a Epic Premier, let's see, Epic is…

[00:27:43] Bill: Moving its AI agents out of the chart and across apps. Well, as we've seen with a lot of these tools.

[00:27:48] Bill: They have connectors.

[00:27:50] Bill: And Epic has introduced connectors, so this will be interesting to see them move outside of just the chart and have connectors into other systems. Wow, that is going to be really interesting. Somebody else told me something this past week that, computer usage, what we were talking about earlier with an agent actually, opening up a browser and running something, Epic's…

[00:28:13] Bill: Stated position on that now is that they allow computer usage from agents.

[00:28:21] Bill: Does that surprise you? It surprised me when I heard it.

[00:28:25] Drex: I think that the,

[00:28:27] Drex: conservative nature that I have come to know Epic for, it surprises me a little bit, but at the same time, it's like…

[00:28:35] Drex: I think there's a group inside of Epic that are really leaning into this, and you know, hopefully they're do… they're thinking about it, how to do it safely. It's not… doesn't surprise me that they're gonna use connectors. I mean, there's MCP servers that they're probably gonna build. I think the conservative part of this is…

[00:28:55] Drex: what data do they allow into the record? And is it marked in a certain way as not coming from Epic, but coming from an external source? And there's probably a lot of those conversations.

[00:29:05] Drex: That are going on, too. I watch… I'm a big fan, I've talked about this before, I'm a big fan of Formula One. The Austin race was this weekend, and the reason the cars can go so fast is because of the engines, but the reasons that the drivers are willing to drive the cars so fast is because of the brakes.

[00:29:24] Drex: And I think there are plenty of brake people at Epic that are gonna keep this from being a…

[00:29:30] Drex: Stand on the gas with both feet and go as fast as you can. That's obviously going to drive an accident. There's lots of people who are tapping the brakes, checking the brakes, and making sure everything's working.

[00:29:41] Drex: Because every time you add an AI agent, the AI agent, very often, is dipping down under the car and figuring out how to cut the brake line so it can go faster, so…

[00:29:51] Drex: Check your brakes.

[00:29:53] Drex: Talk your break.

[00:29:54] Bill: Sarah, let's talk about this one a little bit. So, the fight over who monitors AI goes, AI after Go Live.

[00:30:03] Bill: And the evidence for it at scale. The AHA told the FDA that developers, not hospitals, should carry post-deployment monitoring because hospitals can't see

[00:30:13] Bill: Inside black box models, FDA Digital Health Chief Rick Abramson said generative AI may need to be evaluated like a physician with ongoing competency checks the same week

[00:30:27] Bill: brought scale data. Jefferson, well, and we have a lot of these, so Sutter Health, 5 plus million Abridge notes, 10 AI doc tools live. Jefferson, 34% of 5,000 clinicians active on Ambience. Inova AI reviews went from about 1 to 12 plus a month, with 92 tools in production, AI tools, that is.

[00:30:50] Bill: And, and we're seeing that kind of uptake across the board.

[00:30:55] Bill: So, AI monitoring after go-live, FDA, HA, American Hospital Association, told the FDA that developers, not hospitals, should carry the post-deployment monitoring, because hospitals can't see inside the black box.

[00:31:10] Bill: Sarah, how do you feel about… I don't think I agree with this, but Sarah, how do you feel about this?

[00:31:15] Sarah: I almost, jokingly make up the Office of AI Oversight. Like, you've got a… you know I'm a big fan of what I call, like, ninja squads, or tiger teams. I would have a couple of savants, and that's a… that's a,

[00:31:30] Sarah: That's a thing I would say is a good thing to have in your organization. It's a compliment. If I've got a couple of people who, they are really into this, figuring out how we most effectively create the single pane of glass that allows us to see what's happening organizationally. There are partners who can help you with some of that tooling and whatnot, but you need a couple people who want to really own all of this. Those are the same people who are big into probably my dev environment and or some of the AI utilization.

[00:31:55] Sarah: use case capabilities as well, but I would definitely have people who know how to

[00:32:00] Sarah: watch the organization to make sure that things are happening as intended. So if you do need to pull that plug, or you see some nefarious activity, it's because they have data

[00:32:11] Sarah: that allows them to make some of those decisions that are going to happen more and more. So I would put a couple of people on it who love that kind of work to figure out what are we really seeing? And this is a little bit about us doing our own due diligence inside of our own organization, but I'd rather have some internal watchdogs. It's like when I used to have ethical hackers on my team.

[00:32:28] Sarah: Try… your job is to break into us all of the time.

[00:32:31] Sarah: Like, that's your job, and they were part of our security organization. So, I'm a fan if it's something that you are organically

[00:32:39] Sarah: pursuing to create a greater layout of, I think, ethical usage of AI in your organization. And that's already an accreditation, as an example, you can get from, like, Joint Commission, but I'd add a layer of that into it, and I'd bring some of those conversational partners into the mix.

[00:32:54] Bill: So, Drex, they almost want to make AI a medical device.

[00:32:57] Drex: part of…

[00:32:58] Drex: Look, we've been… this is how you get that, if you keep going down this road. Software companies have fought against being regulated as medical devices.

[00:33:09] Drex: For years and years and years, there's a boatload of lobbyists in Washington who fight against software, medical software, being categorized as

[00:33:19] Drex: medical devices. Medical device companies know how difficult and complicated this is, and how slow… how slow it is to row the boat up that river once you're categorized like that. So.

[00:33:33] Drex: There's a little bit of, like, be careful what you ask for, because I don't know that the FDA is going to somehow figure out how they're going to go fast, and keep breaks at the same time.

[00:33:45] Bill: I go back over the last 3 years and read you everything the AHA asked for on behalf of

[00:33:51] Bill: hospitals. You would think we're the most incompetent people on the face of the earth. Like, every time I read one of their things, it's like, oh, don't… you need to do this for us, you need to do that for us, we can't do this, oh, that's too quick, that's too hard, that's too expensive. And I'm like, oh my gosh.

[00:34:07] Bill: Like, I understand they're protecting me, but every time they talk, I feel incompetent. I feel like, really, we can't figure this out? But I understand what they're saying. They're saying, look.

[00:34:17] Drex: I don't like black boxes either, nobody does.

[00:34:20] Bill: Nobody likes a black box. I don't know how, you know, publish…

[00:34:25] Bill: You know, but what are we publishing? We're publishing the weight models? What are we publishing?

[00:34:29] Drex: I know. There's some transparency between the vendor partners and the health systems.

[00:34:35] Bill: This, this…

[00:34:36] Drex: I know that I need the FDA in the middle of that.

[00:34:38] Bill: This is why I think it's us. This is why I think it's doctors. I don't… do we really… once we get to the point of saying.

[00:34:47] Bill: that it itself needs to be regulated and whatnot. We're on the path for ChatGPT to be a doctor.

[00:34:56] Bill: We are absolutely on that path. But today, what we have is a path where ChatGPT on my side helps the patient, and on the provider side helps the provider, but we see, we know what it is. It is, it serves us, it is something that helps us to do

[00:35:11] Bill: our job more effectively. And so, I take responsibility for what my agent gives me, and what I prescribe, or what I put in front of somebody.

[00:35:20] Bill: But the minute we get past that.

[00:35:23] Bill: Oh my gosh, I just, I just think…

[00:35:26] Bill: I just think… I don't think we want the outcome that we're asking for.

[00:35:31] Sarah: No, you want to do it yourself. Like, if you say, I'm from the government, I'm here to help, like, get way in front of what you anticipate is going to be expected. It's like with the, new security, things that are being voted on, Drex touched on the two-minute drill over the weekend, and it's a matter of

[00:35:49] Sarah: often it's common sense, takes you back to the things you're supposed to be doing in the first place. Don't overcomplicate it, but have something in place. Like, put on your seatbelt, lock your front door. Like, there are basics you can do that allow you to understand what's happening in your environment, and if you follow

[00:36:04] Sarah: Common sense and good protocols, you're gonna be more than halfway there before someone tells you what you're expected to do anyway.

[00:36:10] Drex: Some of this bill loops back around to our conversation earlier about the five stages of AI use, and I think

[00:36:18] Drex: There's the concern about what happens when the AI gets to stage 5 and actually starts treating patients.

[00:36:25] Drex: And who's in the loop, then? Who actually can see what's happening and understands why

[00:36:32] Drex: Is it doing a good job? Is it doing the right thing? Not that we as humans are perfect at it today. We make a lot of mistakes. It's documented over and over and over again.

[00:36:43] Bill: Cheers.

[00:36:43] Drex: It's the concern about what happens in that black box when we get to that point.

[00:36:48] Bill: I have some sinus and ear things going on right now, and the other night, it really hurt to swallow and stuff, so I'm sitting there in my, medicine cabinet at 2 AM.

[00:37:03] Bill: well, I'm not gonna call my doctor. So, pull up ChatGPT, I'm like, hey, here are my symptoms. He goes, well, you know, it's probably this, probably this, and it told me, like, 50 times, you should see a doctor. And so, I will see a doctor. As soon as I get off this call, I'm going to see a doctor. But regardless.

[00:37:17] Bill: It said, you probably have this, take some Tylenol, and I said, well, I have Sudafed. It said, nope, Sudafed's not gonna help you, blah, blah, blah. I'm like, I have Mucinix. Which one?

[00:37:28] Bill: I have Mucinex DM and Mucinex C. It's like, don't take DM, that won't do anything. Mucinex C, that will help you. You should take some Mucinex C. That would be great. Do you have Afrin? No, I don't have Afrin. You should get Afrin. Afrin would be really good for this. I'm like.

[00:37:44] Bill: That's… the crazy thing is.

[00:37:46] Bill: I had a conversation last week when I started to feel poor, because I have concierge medicine, I texted my doctor, but before I texted my doctor, I went back and forth with one of the AI models, and it said, you probably have this. And I went to my doctor, I said, he goes, you probably have this. And it was identical.

[00:38:05] Drex: It's funny, it makes me think about, like, you open the refrigerator, and you take a picture of your refrigerator, what's in your refrigerator, and you send it to GPT, and you say, what can I make for dinner? Your version of that is, I don't feel well, here's what I have, let me take a picture of my medicine cabinet and send it to you.

[00:38:21] Drex: Which versions of these concoctions should I use to feel better?

[00:38:26] Bill: Yeah, it's crazy. But, the, you know, this… I'm gonna, I'm gonna put these in the, show notes.

[00:38:34] Bill: But the Eric Larsen papers are worth reading. Gen AI… Gen AI juggernaut really drives home. Healthcare is an industry, as an industry, he believes, is most disposed to AI disruption. Think about that. He's saying, this is the most…

[00:38:49] Bill: will be the most disrupted industry, and then he came back a year later and wrote his Oppenheimer moment, healthcare's Oppenheimer moment, and he says, AI is the industrialization of intelligence. And he said, you know, he gathers these people together, and he brings in, like.

[00:39:07] Bill: Dario, and he brings in,

[00:39:09] Bill: you know, these leaders from these AI tools, and he shows these CEOs what it can do, and they're like, oh. My. Gosh.

[00:39:17] Bill: Like, because it's… it's intelligence at a scale that, like, is… is completely distributed. Like, you and I have access to the same intelligence

[00:39:29] Bill: that… That the, you know, the world has created over the last thousand years. It's crazy.

[00:39:35] Drex: Yeah, the future is here, but it's not evenly distributed, except now it's becoming more and more evenly.

[00:39:40] Bill: more evenly distributed. I mean, this is the thing we have on social media. We go out and read something, and it's like, did this person write this? There's no chance this person wrote this.

[00:39:49] Drex: I don't know that guy, he's not that smart.

[00:39:50] Bill: He's not that smart, but, but we can all write amazing articles now, because…

[00:39:58] Bill: You know, because AI is sitting there behind us, but we could also do…

[00:40:02] Bill: we can learn how to paint, we can learn how to speak a new language, we can, like, AI can do all those things. That was the point of the conversation, is how does AI…

[00:40:13] Bill: really accentuate our humanity and allow us to be, more of all those things that we are to each other. More empathetic, more…

[00:40:22] Bill: you know, as a doctor, be able to be in front of them more, and listen, and care, and all those things. And I think that's going to be the challenge of our

[00:40:34] Bill: next 5 years. It's like, okay, we have all these tools, this is great.

[00:40:38] Bill: you know, you can… you can use… you can use atomic energy to build an atomic bomb, and you can, you know, build nuclear medicine. I mean, there's… there's a lot of… a lot of,

[00:40:49] Bill: a lot of positives, and there's a lot of things that could be negative, and Drex, you're.

[00:40:53] Drex: It's growing so fast now, too. I mean, it's just, it's unbelievable, like, 6 months ago, where we were, and where we are now. So I, like, I look at 27,

[00:41:03] Drex: And I'm just like, what is gonna happen in 27? It's super exciting to look forward and think about, man, how fast we're going, and what we're gonna be able to do in 2027.

[00:41:17] Bill: It is. And I am glad to be doing it with y'all. We are gonna make these shows a little longer, and we're gonna, probably change the name moving forward from Newsday. Holly's gonna be listening is she's gonna be like, why are you announcing this? Anyway, probably gonna change it to,

[00:41:33] Bill: I don't know, I don't know, we'll call it something like, you know.

[00:41:38] Sarah: the conversation.

[00:41:39] Drex: podcast. Well, it's…

[00:41:41] Bill: It's the principals… what did you call it, Drex? The principal's… Whatever.

[00:41:45] Drex: I don't know, eavesdropping on the… yeah.

[00:41:47] Bill: eavesdropping on the principals, but essentially, that's what it's gonna be. It's gonna be us, as we go out and talk to these various healthcare leaders. We're looking at what's going on. This is just you getting to look behind the scenes. I tell people, they're like, oh, from time to time, people will tell me, boy, you're really smart.

[00:42:05] Bill: Which, I don't know how to respond to that, but what I will say is.

[00:42:08] Bill: I'm in a lot of rooms, so you get to hear what this health system thinks, and what this health system thinks, and what this… and so I sound smart, because I just take the best of…

[00:42:19] Bill: The 70 rooms that we're in, and… and…

[00:42:22] Drex: Totally agree. I'm a great router, like, I'm a great, you know, I'm an awesome ventriloquist dummy.

[00:42:30] Sarah: than Congealer, doesn't it?

[00:42:32] Bill: Ew. Never mind. Yes!

[00:42:37] Sarah: A.

[00:42:37] Bill: yellow of some kind. I don't know what you're doing there. It's scary.

[00:42:41] Sarah: Monday, you get what you get.

[00:42:43] Bill: Get what you get. Well, hey, thanks guys, and thanks everybody for listening. That's all for now.