AI Ready Data: Donna Roach and Utah Health Innovation

[00:00:05] Bill: All right. It's The 229 Podcast, and today we have a special podcast. I'm looking forward to this one. We are going to be talking to the team, the Innovation, Infrastructure, Data Operations team from the University of Utah Health System, and I'm looking forward to, you know, diving into this, because I've heard some really wild stories about the NVIDIA GPUs, and I'd like to get that out into the wild and have that conversation.

[00:00:34] Bill: Donna, we're going to start with you. Enterprise Chief Information Officer, newly appointed Enterprise Chief Information Officer for University of Utah.

[00:00:41] Donna: November 1st.

[00:00:42] Bill: Over overseeing both sides. Congratulations on that.

[00:00:46] Donna: Thank you. It's

[00:00:47] Bill: exciting. Appreciate you.

[00:00:50] Bill: You know, it's interesting, because in some areas, there's a lot of overlap, but there are some new systems you're going to have to learn on the university side. Clearly, the healthcare side, I've been a university CIO, and I've been a healthcare CIO. The healthcare CIO side is much more complex than the university side. So, it's hopefully a pretty seamless migration. I'd love to, I'd love for you to introduce the team we have on here, and give us a little, and maybe they can introduce the work that they do at the university.

[00:01:29] Donna: Okay, great. Yeah, thanks, Bill. So, the most, here's the thing I'll lead out with. These are the smartest people in the room, much smarter than I am. And so, I so appreciate everything they do, how they support me, how they'll sometimes look at me when I say things, and they're like, hmm, shake their head. I'm like, okay, okay, maybe overstep my level of understanding. But they honestly are some of the best people I have ever worked with in my career, and I've been doing this for over almost 40 years. So, I have Brent Elieson. Brent, I got to know Brent early on at when I came to University of Utah Health, because he really led out in our virtual care and telehealth network. And he does some amazing things in the innovation space, specifically over at Hunts, not Huntsman, but Nielsen Rehab. And I will let him talk about that, because every time I'm over there, or I bring people over there to show them the technology that they do, people are in awe. And I, you just can't, I can't imagine sometimes the brainpower that went into creating all the adaptability and technology that supports people when they are the most vulnerable. So, I'll let Brent talk about his work. Shawn Halladay is an amazing leader. He came out of the infrastructure space, and I kind of stole him away from Jim Livingston, who's our CTO, and Jim probably still hasn't forgiven me. But I asked Shawn to lead out in our AI space, and all things AI, and sometimes he gets, you know, we're in the healthcare space, but we also do a lot of audibles and get pulled into the university space. So, Shawn has been gracious enough to step into things too, to help out at the university level, especially we have a brand new enterprise CFO, and he's been helping out in that area. So, very appreciative of Shawn and all of his knowledge, and he's the one that can probably tell you more about the NVIDIA environment that we stood up. And then Reed Barney. Reed Barney, I got to know Reed when he didn't really report into ITS. He was our head of our enterprise data warehouse on the clinical side. And we work very closely. Reed has always been very gracious in terms of bringing me along, helping me to understand the wealth of information and data that we have in our clinical enterprise data warehouse. We got to a point about over a year ago now, right Reed, where we said, we need a chief data officer, and it became very apparent that how we had structured things were okay. But as we were doing more progress around InterSystems and things with Databricks and stuff, we needed a chief data officer to lead out in this space, and work hand in hand with our chief insight officer. And so Reed participated in a national search, and he stood up against some really strong candidates, and we selected Reed Barney to be our first healthcare chief data officer. And so he will talk more about the sometimes, he uses sometimes terminology like OMOP and things that I'm like, okay, now tell me again what that is.

[00:05:28] Donna: Honestly, Bill, there are three of the smartest people right now on this podcast. And when you combine their brainpower together, it is just amazing. And we try to talk through a lot of things, sometimes in a very casual setting, sometimes in a project setting, sometimes in a strategy setting. And these are all the main contributors to that strategy.

[00:06:27] Bill: Yeah, I'd love to hear from each of you as way of introduction, really about the area you focus on. I'd like to hear how you guys interact, too. How often do you get together, and how much does your stuff,

[00:06:42] Bill: how much does it rely on each other, how much are you working together, and how much of it is sort of siloed where you're advancing the thing that you're responsible for, or is it all tightly aligned? And, Brent, we'll start with you.

[00:06:59] Brent: How often do we interact with each other?

[00:07:02] Brent: You know, I'm hard pressed to think of any activity or initiative I'm working on that doesn't involve both Shawn and Reed. It seems like there's data involved in everything that we do. And AI is an increasingly important component, and often the main building block of what we're trying to leverage to accomplish our business outcomes. So, on a transactional basis, we are interacting all the time. And we do have regular meeting cadences set up to try to keep the more strategic conversations going as well.

[00:07:52] Bill: Yeah, you know, guys, I started with Brent, because if his stuff doesn't work, you guys don't have anything to do. I mean, like, it all sort of breaks down from there. Shawn, give us an idea of what you do and how you interact with the team and what you're looking for from the team.

[00:08:10] Shawn: Yeah, yeah, AI is really interesting, right? It's the first technology that landed in everyone's pockets pretty much out of the gate. And so it wasn't really a gated or permission -style technology or service that we've dealt with traditionally from an IT perspective. And so you've really needed that influence of how do you modernize and innovate in an organization. And I see modernization is taking those, you know, current processes, teams, you know, operational needs and take them to the next level and do so by integrating AI. And a lot of that is done by, you know, shepherding the data through and changing, you know, you get into like a headless workspace where headless is, you know, you're needing that software layer less and less all the time. And you're just providing those AI services that get access to that data. And that's really where, you know, bringing Brent in for some of the process and the innovation to shepherd things across where traditionally you go through these intake processes that can be a little cumbersome, right? There's much more agility needed in the space. And so keeping it all tied together has been a big effort, but it's starting to pay off, you know, as we get into some of this space and starting to see good results and making sure that like I said, those traditional gates or ownership is really not there as a barrier anymore, but it's more about access and use of AI and then how that plums into the rest of the environment. So it's just like a high -level approach. We want to enable the system, right? We want to make sure we're making good strategic decisions that are secure, that we enable the system. And that means a lot when you think about enablement from a solution, right, or a traditional technology is like, how do you get people access they need to do their job?

[00:09:55] Bill: I like the way you talk about it. I like the way you talk about AI, because it is, the conversation I'm having with people is, what would a headless environment look like in healthcare? And I know on my PC, on my desktop, because I'm not in a regulated environment and whatnot, I rarely leave my AI agents anymore. Like I don't log into, I don't go to Google Drive. I don't go to, I don't go to these things anymore. I just ask for it. And it comes to me in the window I'm sitting in. And I think that represents, you know, obviously, it's going to move a lot slower in a regulated environment and whatnot. But I think that represents what we might be looking at in the future in healthcare. Reed, which brings us to you, all that data, he has nothing to deliver unless the data is right. Talk a little bit about, on your role and how you're setting the organization up for success in this ever -evolving healthcare world that we're in. Sure.

[00:11:04] Reed: I think that one of the, data is kind of the,

[00:11:11] Reed: if you think about AI, there's kind of, it kind of sits on two things. It sits on the model on the right-hand side and the data on the left. And without the data side, the model will end up hallucinating like crazy. And so it's really my task and been tasked this with the, by our CEO to make our data AI ready, make it trustworthy so that it can be used in agentic workflows and by people as well. I kind of view our increasingly, we will be changing our customer, the people who consume our data from people to agents, right? And so we have to make sure that it's, that the data, that that ability comes with the data, that it's AI ready, that it can be used by agents as well as people. And that, that it's, it's set up with the appropriate foundation and that there's shared metrics where shared, shared knowledge layer, shared semantic layer, all those things that make it AI ready. So we have, you know, not only do we have AI as our customer, we have many other customers as well. And so it's, it's a pretty big task to set up a data ecosystem for a healthcare environment, let alone an academic healthcare environment where everyone wants to do their own thing. So a lot of flexibility, a lot of agility, but yet governance and trying to make sure everything is aligned and connected. It's a, it's a big task.

[00:12:55] Donna: so Bill, I'm going to, I want to mention something because for me, this was a big aha, working with Reed and, and actually our CMIO, Dr. Vivek Reddy and, and where, how we look at our clinical enterprise data warehouse and how it is not just, to call it a data warehouse is almost a misnomer in a way. It is part of our clinical decision making on a regular basis. So the data, the way we replicate, we pretty much have Epic replicated into our enterprise data warehouse, along with other people. And it's an integrated component to clinical care every single day. People don't realize sometimes they're like, well, am I in Epic or I'm in the enterprise data warehouse? It's like, we don't, we don't spell that out. It's so integrated together that it is a organism that's supporting the clinical enterprise. And I think we've evolved to that because we've been up on this structure for over 30 years now. It was a strategy that really came out of Jim Livingston's area to create this structure. And then Reed has been kind of the, the custodian and director of how we have enabled more and more uses of this very powerful clinical tool. So it's, it's kind of different. Like I've been trying to describe this to the university. That enterprise, clinical enterprise data warehouse is an integral part to everything that we do. On the university side, academics looks at their EDW in a very different way. It truly is a data warehouse versus what, how we utilize the EDW on the clinical side. Yeah, we view ourselves as part of the care team. So yeah. Absolutely.

[00:15:17] Bill: Yeah. So you're describing sort of like, they view it as somewhere to put data in storage for use at a later time. And you guys see it as more of an ongoing transactional. I mean, it's a data warehouse, but it's an ongoing transactional system. And you, I mean, obviously, you know, one of the things we talked about in the AI world is context. And context comes from the data. And I think one of the mistakes that people make, this isn't really about me. I want to take you guys through this conversation. Actually, I'd love for you to participate in this conversation because it's, you know, context, context is so important. And the, you know, AI relies on it. It's how it keeps from hallucinating. It's how it makes sure that it validates its answers against things and whatnot. I think a lot of times these conversations start with, hey, what tool are we going to use in AI? Or, you know, what are, you know, what model or what, and all these other things, instead of starting really where it needs to, which is what is the, you know, what is the use case that we're trying to solve? What is the data that supports that use case? What is the context we can draw from that? How can we determine what needs deterministic versus the probabilistic nature that is healthcare? We're seeing all these parlor tricks out there. But in your world, I don't think you can rely on these parlor tricks. You have to create, you have to create these models, especially in the clinical care setting that are delivering insights in real time. So help me understand this journey. So you guys are all sitting there. Brent, you're looking at controls and what tools are these people bringing in? I'm sure it's an academic medical center, so people aren't trying to bring in their own tools all the time. It's probably all very standard and very easily accessed. You don't have to worry about that. And Reed, there's probably a very defined set of people that want the data. It's not like everybody wants data and that kind of stuff. And Shawn, you probably have the longest list of requests for things coming down the pike. I'd like to hear how you guys sort of address some of these things. How does the conversation go? How did you organize it so that you could address all these things people are struggling with? The number of projects, the controls, and the quality of the data going in and out of these systems.

[00:17:44] Donna: So one thing you have to put in context of how we're set up. So when I first came to the University of Utah Health, we had what I would call more operational governance groups like Application Advisory Council, which really helps run a lot of the epic tools. And there's EHROps kind of that feeds into that. Reed and Candace kind of oversaw the data governance. And then there was kind of an infrastructure, a TIAC technology and infrastructure governance. What was missing a little bit was this overall thing that kind of pulled it all together, like a higher level of governance. So we created a digital enablement committee, that's DEC for short, and that helped set the roadmap, set the priorities, set the focus of what we were going to work on, and filter things down to some of those secondary governance groups. But it just wasn't a one way. It could come back up to our governance structure, our DEC, in the sense of we may not have given good direction, we may be in conflict with one another, or more like where does it sit on the roadmap. They discovered something that said, you know, we could push this out, and this needs to be like, one needs to be promoted initiative, and another one needs to be demoted. And so it's a very iterative, interactive. So having that form of intake, DEC took strategy and set the roadmap. But then from there, it was these more operational groups that got into the nitty gritty of everything that's going on. And it's funny that you bring so, you know, if you think about project management, and how do we manage all the projects, you know, at one point, you could say, yeah, we were very much a waterfall organization. The last four years, we've been on what I would call a safe agile transformation journey, we actually kicked off our PI eight. Today, I was over there with probably 300 plus of my best friends, not only ITS leaders, but operational leaders setting all of the next initiatives and what what are the stories that are going to be created, what are the epics that will come out of that and kind of the train that will be focused on those all interact with this within our governance structure. Now, fast forward to today, right, because we brand new CEO, Dr. Carter King from Mass General Brigham, very, very much innovation, like in the forefront, all of these gentlemen know that if there's going to be somebody driving innovation, it will be Dr. Carter doing that. And he create he helped us create what we call our technology and innovation strategy group. So TISG kind of sits with deck. And sometimes they compete. Sometimes innovation, you know, is driving something, but we, we kind of are working on morphing it together. So it still has to hit the roadmap, right, because innovation has a faster spin cycle and how we might scale it could be different. What Shawn and Brent work with in terms of, you know, the intake process of those things. But as we get more and more down an operational component of an innovation project, it has to come back into deck or any of these other operational groups. It's not perfect. We're learning as we go. Sometimes like, you know, the old adage, we're, we're putting the plane together while we're in flight. I definitely feel like that sometimes at the 30 ,000 foot level where I'm little gasping for breath. But it, it people, people at the you really work to collaborate and compromise and get things done together. And so we may not always agree on initiatives or somebody has something that came from innovation. And they really want to push it and they, it needs to get done and it needs its epic connectivity. We still work through the engineering process of doing that, but still recognizing how important that innovation may be to the organization. And, you know, that's kind of the constant balance game that we're, we work on. So that's, there's a lot of intake coming from all different avenues, trying to listen to different chairs and clinical like the deans of the different schools in terms of what's important to them. I mean, Brent and Shawn kind of get smack in the middle of it from the innovation side breed working with our, our chief insight officer is smack in the middle of trying to create more insights for the organization to help with strategy and focus on strategy.

[00:23:36] Bill: The, I'd love to hear from the three of you on demands. So creative deck, deck sounds really, who's on deck, who's on deck, but who's on the.

[00:23:49] Donna: There's about 20 members, very physician oriented, I would say we have about six chairs, three operational leaders, a few of my ITS leaders are on there. Our chief AI officers on there. So

[00:24:08] Bill: they're looking at digital initiatives, right?

[00:24:12] Donna: Digital, but the full digital roadmap. So I look at it from the sense of honestly, digital is just a word. It really is the ITS strategy for the organization. It's the ITS roadmap. And there are digital components in all of it.

[00:24:34] Bill: So, yeah, so that's, that's interesting. So when you're looking at, let's just take an example. When you're looking at a new Epic AI feature, will that be handled below them or will that end up bubbling up to them?

[00:24:46] Donna: Well, Shawn, you answer that one.

[00:24:48] Shawn: Yeah. That's one where we've, we've looked at where do we balance this out with innovation versus kind of your modernization of a current workflow. So it's like with Epic, because it's already in process, in workflow and has the integrations already there, the governance already there. We've opted to move that modernization effort in with the process that already exists and have awareness and communication and training adapt to that. And so we have a, we have an AI team that does literacy and training efforts that now coordinates with the other, you know, so -called training other teams that have done that traditionally. And they're both modernizing together and they're working on that collaboration to make sure that's fit for use on that space. The other spaces that come out, it's like net new requests for AI, right? We actually have an AI intake process where you can submit an idea, whether it be like an email or just a few words or whatever, and we have that fronted with AI and AI will actually walk you through and automatically triage that request. And that triage goes against our policy and some other forms and our strategy for the organization that it comes out and ranks your use case based on the risk, based on priority, based on strategic fit. And then it goes into this pipeline with other governing connections such as data and, you know, Brent is going to shepherd that through from innovation, you know, external and internal and these other things. So that's really how we started to adapt to that, that we provide a bit more freeform space for those things that are net new, and that we also integrate that modernization effort for things that already exist, that they don't operate in the same space because we want to modernize those teams and make sure that they're integrating AI as part of what they do day in and day out, and that's not something that's handled outside of their teams, that they modernize their workflow and what they're doing on a daily basis.

[00:26:41] Bill: So the intake right now is through an AI engine that walks them through and makes sure that they document. I remember that process, like we would have a team that would look at all these things and go, well, we need more ROI numbers, we need more of this, we need more of this, and it would go back and forth. But now the AI just walks them through, and then it doesn't go straight up to the deck, it goes down into the normal intake.

[00:27:06] Shawn: Yeah, that's right. So it'll give you a score and sort of automatically prioritize it, and it will score it based on what the human input was, but then AI will actually add or subtract from that scoring based on the policy, based on strategic initiatives, based on other things that it's aware of and that are happening. And so you can see the score adjustment that tells you why it's scoring differently. It could be data risk, right? It could say this falls outside of Epic and these kind of things, and so this is actually a policy risk, policy and data risk. And so we know that's automatically flagged for additional triage through data governance. But that's been very helpful, and it's been great. You know, like CIS controls and those kind of things, I mean, that's basic following of what it is, like, what do you have and what is it doing? You know, just start there, and then that's been all different.

[00:27:56] Bill: Is that through an AI model or is that through like ServiceNow or something?

[00:28:00] Shawn: Yeah, we actually, we looked at a few different solutions for that, and we wound up building our own. We did, we used AI, right, and coded a site, and so we have an internal site that is secure so it can handle restricted data, and it's just internal for us. And so we actually wound up building it after doing some trial and error and having kind of a bake -off amongst different tools and services, and it's worked out great.

[00:29:12] Bill: So, Brent, this stuff comes to you. I assume you were there for the old process and there for the new process. Does this make it easier that the stuff comes through not half -baked but fully -baked, prioritized, and ready to go? And does it help with the fact that the list is as long as you're on? Well,

[00:29:31] Brent: yeah, so it's interesting that the, you know, this traditional intake method worked, it worked fine, but it required a lot of people time to do the analysis, a lot of the back and forth. And what we've attempted to do is to more strictly proceduralize the back and forth and the information gaps, the concerns,

[00:30:04] Brent: and to get good at getting consistent data to inform the process. And when there's gaps or there's something missing,

[00:30:19] Brent: having AI tell you, you know what, you've got this security compliance concern here or there, it's received much better coming from a machine saying, hey, you know, we've got these guardrails and we need to work on this area than it does from a person. And when we can proceduralize this, we can also automate it, as Shawn was mentioning, how we don't have to wait for meeting schedules, you know, between very busy people to align, to have a conversation, to release a gate, to continue to move forward. This can be automated and it can happen in moments rather than months, many times, to work through some of these complex issues. So it's, we're not there completely, but we're making great strides in making a lot of progress towards automating.

[00:31:24] Bill: You know what I would love about it is the AI is educating in the other direction, right? So as they're filling it out, I would assume they're getting feedback that normally you'd have to tell them like, hey, wait a minute, man, there's some data issues here, there's some security issues, whatever. But the AI is just saying, hey, you're asking for data over here, you're asking for, this might take a little longer and your, you know, your risk score is a little higher.

[00:31:49] Shawn: One of the things that was really interesting, just kind of tagging onto that,

[00:31:54] Shawn: Donna, you know, had requested that we add an executive sponsor as a requirement. And at the time I saw that I was like, yeah, that's important. I did not realize how critical that is to ensure before they get through that, that you have a documented executive sponsor for every AI intake. Because what you're asking the organization to do is commit time, resources, and effort just in the review, but not only the standing up and use of this. And now the AI can stand up services and integrations across the organization. And as we mentioned early on, everyone has access to it at some level or another. It is critical to make sure that you are aligned and that your executive sponsors understand what is being done in each of their areas that we're following that path. That has made a huge difference for us. And I thought that was just a great call out, just with the knowledge that Donna has and like, yep, that's probably one of the key elements that has made this so

[00:32:50] Bill: successful. It's not knowledge, it's experience and wounds is what it is.

[00:32:55] Donna: it starts. But the other thing too is we don't take people out of the loop in totality. We have a regular stand up where we go over some of these. Sometimes if we have questions, we may invite the requester. Sometimes the conversations can get very heated about what, you know, hey, this was approved. Why aren't you moving forward with it? Why aren't you doing this or something? And, you know, it's understanding it against all the other, I don't know how many are in the queue now. The last I looked, it was over 60 use cases. It's probably more than that now. But you still have to have kind of this little stand up meeting to go over things, handle questions. You know, people might see what was outputted and they totally disagree with it. And it's like, well, here, you know, we have people who work with them, but it's good to get a lot of the experts, some of the SMEs in the room to say, okay, here's why it got this score. Or here's why when you said the problem is this and you're defining the problem, that really isn't a problem we need to solve for now. Or you didn't define the problem. You gave us a, I want this tool, which, okay, what problem are you solving for then? So it does, you still have to have the people in the loop and the interaction with them. One of the things, Shawn and team have done a great job. We stand up office hours. And in the beginning, we did clinical office hours and very focused on in the health system. And then I pulled Shawn's team into our kind of the CFO office hours around AI to bring that them into the kind of the iteration of what we do. Really very different. I mean, their maturity level, what they're looking for, where they sit with their data constructs. And there may be on four different spreadsheets and it's like, whoa, wait a minute. How do you apply AI? AI is only, like you started out, is only as good as the data, right? I mean, read his testament to that. And so you learn a lot by having these very kind of open office hours where people can bring in their use cases and we can discuss them and have some great dialogue with people. And anybody can come to them. And we've seen anybody come to them and it's kind of.

[00:35:46] Bill: They don't come with Microsoft access databases anymore, right? Just spreadsheets.

[00:35:51] Shawn: Spreadsheets primarily. Yeah.

[00:35:53] Bill: Smart sheets.

[00:35:56] Shawn: Smart sheets is a

[00:35:57] Bill: good one. You know, Reed, we have this conversation and we're going back and forth. And I think some people are listening to this. And I only know this because I'm in a lot of these rooms talking to people throughout the year. They would be jealous of where you guys are at.

[00:36:11] Bill: I'd like to hear from you what you had to establish in the EDW in that whole world to allow for the flexibility to support all these different use cases and all these different models. Because when I, you know, Don and I sit in these rooms, we hear these people and you can describe some of these things and we're like, oh my gosh, like we're, we can't, we can't do that with our data. Like, I don't know how you got to that point. I'd love to hear a little bit about your journey and how you, how you think through it to get it to the point where you can support so many things.

[00:36:44] Reed: Yeah, sure. So if you think about healthcare data, it differs a lot from maybe other arenas, right?

[00:37:01] Reed: There's. It's completely unique in any data arena. And so in order to make sense of that, we've had to, what our approach is, is a data product approach. So the idea is that you are able to stand up reusable data products to help with organizing the data, data products or data marts as they were commonly known in the past. And on our team, we've kind of reorganized around this concept where you have a data product owner who owns that product. It owns all of the pieces of that product from the data pipeline to the semantic layer, the knowledge layer, the data contract, all of the things that make it AI ready or gold tiered in your, in your medallion architecture for data. And so that, that's really our approach. And we have what we call our foundational data products, which you might think of as encounters or labs or patients, those, those very foundational things. And from that, we build another layer of data products, which are more domain specific, like you might have a pregnancy domain or oncology domain. And, and that those, those are reusable data products for those specific departments or domains. And then beyond that, we, we build more consumption level data products, which might be for a Tableau dashboard or for a conversational analytics, a Gentec approach. And so all of these things, building reusable products and making sure that they're AI ready. That's, that's really, that's really our approach to this. If you think about one recent example that we had, that kind of illustrates where us and the AI team are kind of part of this part of the care team. So there was a recent example where we were trying to track down with our infection prevention team, some, some different organisms that were, could be affecting some of our patients, right. And in order to do this, there were a couple of different approaches. We could take the AI approach and throw a bunch of data AI and have it kind of sift through to try to track down these organisms, or we could use more of a predictive modeling, discrete data approach. And there were two camps, two camps, right? One wanted to go one approach, one wanted to go the other approach. And my task was to make sure that they had the same data foundation or data product to work off of. That way, they weren't working off different data, getting different answers. They were working off the same data product that would make sense so that there was like a shared context that of the situation as well as the semantic layer. So that was kind of one cool thing that happened to us recently, where we set up that same shared data product, and we were able to set it up rather quickly because, you know, we're now using AI ourselves, right? So the time to market for something like this in the past would have been three months to six months, and we've had to increase that and, or decrease time to market so that our end users and the AI teams can use these things in a much quicker and faster way.

[00:40:30] Bill: But your data teams are also using a set of AI tools around building that out. And I wouldn't say we have all the answers to this, right? We're still narrating and developing through this. But it's been a ton of fun, for sure. With the interviews I'm doing, I'm trying to figure out which approach people are doing. Are they embedding AI into the applications, or are they bringing AI to function on top of the data? And the answer is, yes. And the answer is what they're... My challenge with this is, I think if you do both, it's going to be really expensive. That may or may not matter in some cases, like you want it embedded in the workflow, and to see, you know, all the things that are happening in the workflow itself. But I'm finding the more... The more I deal with the agents and the bots on the outside operating on this data, I'm sitting there going, I'm actually the code we've written here. I'm pulling AI out of the applications more and more, because it comes... It's like the workflow is built into the instructions for the bot, and it comes in and it grabs the data it needs to. It understands the semantic layer, it comes back, and it does stuff. Again,

[00:41:46] Bill: you operate... And workflow

[00:41:47] Donna: isn't always consistent. So you have to still... And this is where it gets very complex in healthcare. You know, Reed described, you know, the 40,000 tables, and then the indexing of all those tables. And that's just epic. That's just epic. But you find inconsistencies that are just inherent in because it's different workflow, right?

[00:42:17] Bill: I was talking to an AI company that works in the medical record, and I said, what's the biggest challenge you face right now? He goes, the medical record does not agree with the medical record. Because that's our biggest problem. Because the problem sets don't agree from... The med rack doesn't agree from... And he's just like the medical record itself. He goes, so we're sending AI in there to figure out what the longitudinal record should be. And it's like, should it make a decision based on certain things? And even humans would struggle with it. They would just throw it all out and ask the questions again. What meds are you currently on? Let's go ahead and make a new list.

[00:42:52] Donna: Well, and so one of the things, and Shawn's worked on this with our chief health AI officer, Dr. Ken Kawamoto. He's created kind of an AI workbench that is embedded within our epic environment.

[00:43:09] Donna: And that requires, I don't know, how many tables can he pull from on that? It's maybe like 40 tables, right? When you pose a question, when the physician sits down and they pose a question, they can say, okay, pull from the encounter list, pull from the meds list, pull from nursing notes, physician notes, in order to inform the answer that they're looking for in that request. And so it does help narrow it down, but you'll still see inconsistencies. And that's why kind of the physician or clinician still has to be in the loop to say and look at and be able to tag and go, okay, this is where this is pulling from. And this is why maybe two years ago, this person may have had this diagnosis based on this lap result. But now we're repeating that lap result. And now those two things aren't meeting up together. They're coming in conflict. But it gives more richness to the physician to decide and help and treat, because would the physician have ever looked back two years ago? Right. And probably not.

[00:44:29] Bill: Probably not. Not given their schedule and what they're requiring.

[00:44:32] Donna: No, no. And yeah, I mean that and it's, you know, we've had physicians who have been using this tool now have said, oh my gosh, like it changed how they were treating that patient or it changed how they're treating their panel or looking at they're starting to learn about, oh, I need to ask this.

[00:44:54] Bill: Far far away, the most powerful thing I can do is it can it can go through mass amounts of information and bring it forward and summarize it better than we ever could. I mean, and some medical records are literally binder and binder and binder deep. And it's much for the physicians. Now, I tease this at the beginning. We sort of have to talk about it because you got into NVIDIA GPUs because of a grant. It's not like you guys sat around one day and said, you know what, we should build out our own super compute. We're going to compete with Elon Musk and Sam Altman. We're going to all this other stuff. But but you you you got this grant. You looked at it and you decided, hey, this is the path that makes no sense. I want I want you to walk us through that because I think there's only two health systems I've run across and you're one of them that actually has some NVIDIA GPUs. And they're saying, hey, there's a lot of value in us doing this this type of work locally. So Donna, maybe help us understand the process for how you got here. And then Shawn, talk about what what has been built out and why it makes sense for I guess you call it the you. I'm not from Utah. So that if you call it the you, that's what I'll start

[00:46:06] Donna: calling it. OK, we call it to you. So and two different things going on here. We have been very lucky to have the Huntsman family who provided a lot of funding in for development on our responsible AI Initiative at the university level. And that's where I think they they gave over 10 million dollars to set up this responsible AI And and that has progressed. But what I will say is Dr. Carter over on the health system, when we started looking at do we do this, you know, do we do it kind of in the cloud, whether it be with AWS or any of the other services that are out there or do we build out our own NVIDIA environment, Carter gave us, I don't know, in the beginning, it was like, what, two point five, three million. Right. He's up. He's given us more money now. And we so we started through his again. He he sits on our TISG committee. We made the request of the funding. He gave us that money. I took some monies out of my capital budget and helped supplement. And we started out with what, B's or H's. I forget which one. You know, the branding and the type and the style. And yeah, that's that's where we started foundationally.

[00:47:46] Donna: Yeah. And so that's that was the start. And then how we created we kind of created a concept of having a kind of a clinical lab space, which Dr. At the point where we're going to be standing up kind of our research lab with those in the NVIDIA environments and we'll be standing up a a academic or more non academic but more back office operational one. So like I can take some of the CFO initiatives and push it through and start to help look at things utilizing AI based on their use cases. So it really did start with we made the pitch to Dr. Carter. Dr. Carter gave us monies and he continues to say, if you need more NVIDIA chips or, you know,

[00:48:53] Donna: you create the use create the

[00:48:55] Bill: nobody can find them around the country, but you guys have

[00:48:58] Donna: some. Well, we do. And it's kind of funny. Poor Shawn, he gets beaten up on a regular basis about where's my B's, where's my, you know, you're like, oh my gosh, you know, people, this is not like going to Lowe's or Home Depot and picking it up off the shelf and putting it in. And they're big energy consumers and they have to be segmented different in our data center and there's a lot of logistics that go in setting up these devices so Shawn can speak to it better than I can.

[00:49:35] Bill: So Shawn, is this just an inference engine? I mean, a big inference engine that

[00:49:39] Shawn: you're... Am I

[00:49:42] Bill: thinking too

[00:49:42] Shawn: simplistically? Yeah, no, I mean, generally you're creating like a token factory, right? That you basically have your inputs and it's going to run that through and then give you summarization out and kind of that inference. We want to expand on that as well, but I mean, primarily that's the use case because you've got your integrated AI services with Epic and other applications. You have your off the shelf products from open AI and anthropic that you connect to through API or have the bots and agents come in. But there was really this gap of opportunity to say, how do we create net new services that are complementary to other things that exist in our environment? And that's where this came to be. And so we're starting to see some of the research elements come in, the summarization, crossing that data element that Reed has been so instrumental in because he's modularized the data that we can actually plug in certain data sets for specific use cases. And that we've created an ecosystem that has a foundational base of inference and kind of tokenization of our data, but then it goes up to specific verticals, whether it be like pathology or whatever the case may be, that we can do fit for use services in each one of those based on these GPUs so we can dedicate a specific resource to each one of these verticals and have outputs in a clinical space. It's worked very, very well, but it does sit apart from our other infrastructure to make sure that we maintain sovereignty of the models and what we're doing kind of a sandbox environment before we move it through into production.

[00:51:09] Bill: So you have research, clinical and operational use cases. Can you give, I mean, we're close to the end here, but could you give like an example for each?

[00:51:16] Shawn: Yeah, yeah. So right now, like Donna mentioned, the research is pretty early on. So I think those are just kind of stubbed in, but Dr. Kawamoto is backed by that environment, so that's something that's very familiar to him. So those are just kind of elementary at the point, right? Very, very early on. The operational space is something that we've used in some of the financial and other workspaces in that space as well, bringing that data forward. And that's been more the assembly of the data and basically saying, what do we have and what is it? And that's been very helpful in that one. And then the clinical summary has been transformative, as that one is touching on production style, where we've actually gone with patient records, patient notes and those kind of things and brought those forward and had additional insights for treatment plans and awareness of things that are happening in the patient note. And that's been remarkable, is what I'll say for that space. That's our most mature environment.

[00:52:12] Donna: Yeah, so one of the other clinicals, so the biggest clinical one is this AI workbench. We also did something with our Heal at Home program, where they were trying to identify patients early on in their treatment, in their inpatient stay that could be discharged to our Heal at Home program. So two physicians gave us the criteria and we've just built upon it. It probably, and guys correct me if I'm wrong, I think the last we looked, it probably saved about 70 patient days since its inception. So that's

[00:52:53] Shawn: just access,

[00:52:54] Donna: right? That's an access issue. On the research space for Huntsman, we've worked quite a bit on helping with trials. Recruitment, patient recruitment, because that's a very heavy intensive,

[00:53:12] Donna: it takes a lot of navigator people time, and so it's like serving up all the data that they're looking for, trying to serve that up and giving that to the navigators or to the clinical trial recruiters. In the, I would say, in the research space, there's a couple of grants that we're going to look at trying to track better outcomes out of those grants, that they were just doing it manually. And so they've already got the framework set up. We haven't done that one yet, but that will be really key. Some of it is just around kind of battered women and tracking on that. On their health and how, you know, you can kind of protect and be more proactive. And then...

[00:54:01] Bill: clear, Donna, you're going to be going through governance for all of

[00:54:03] Donna: these, correct? Yeah, yeah. We already, some of them are already, like that heal at home one, has already been operationalized. So it's already owned out in the environment. And yes, they constantly are adding on, but it's operationalized.

[00:54:22] Bill: But that's what we're seeing, the art of the possible. People like see it, and then they go, oh, man, we could do this, we could do this, we could do this. And just most organizations across the country are just getting flooded, and they don't have a mature process. And I love the things that you guys talked about this time around, just the process really does. And the process and the architecture, the word we don't talk about much in healthcare. But Reed, the work you've done around architecture enables Shawn to do the stuff he's doing. And again, Brent, I come back to, if your stuff doesn't work, nobody's works. And Donna, I appreciate you introducing your team to the 229 Podcasts, and I appreciate the time. But I want to honor your time, because we are at the end of our time. So thank you.

[00:55:11] Donna: Thank you very much. Great talking to everybody.