3200: CitiusTech on Trust, Transparency, and AI in Healthcare

[00:00:03] What happens when AI begins making autonomous decisions in healthcare? How do we ensure that these very systems are transparent, secure, and trustworthy? Especially when patient lives are at stake? Well, in today's episode, my guest is from a company called CitiusTech, and they are a company at the forefront of health tech innovation. So together today, we're going to explore the concept of agentic AI.

[00:00:32] These AI systems designed to make complex decisions autonomously. But explore their growing impact on healthcare, and how agentic AI is driving efficiencies, streamlining complex workflows, and supporting clinical decisions. All while highlighting the challenges of transparent... And yes, we will highlight the challenges of transparency and accountability in these automated systems. But I want to focus on the solutions.

[00:01:00] How we build robust trust frameworks that ensure fairness and ethical decision-making, and the opportunities that come with such transformative technology. So the big question is, how can healthcare organisations leverage these advancements without compromising patient trust? And actually, enhance patient care? And what frameworks are needed to ensure that AI remains auditable, secure, and compliant?

[00:01:29] And ultimately, how can all of these technologies that we're talking about every day on this podcast translate into better patient outcomes? That is the big goal here. And we're going to do all that by going beyond buzzwords, exploring how AI can unlock value-based care and personalised healthcare experiences. But enough scenes setting from me. Let's get my guest onto the podcast now. So a massive warm welcome back to the podcast.

[00:01:57] We first spoke last summer time, where the weather was much, much warmer. But for anyone that missed our previous conversation, can you just tell the listeners a little about who you are and what you do? Yes. Rajen Kohli. I lead CTS Tech. We are a health tech company, and I've been in this role for two years now, coming to two years. And I've been in the tech services industry for almost 30 years now. So look forward to talking to you today, Neil.

[00:02:26] Yeah, it's a pleasure to have you back on the podcast. We find ourselves right in the middle of Q1 2025. And I've got to ask, what emerging technologies have you seen this year? And certainly since our last conversation around healthcare and health tech. And what are the biggest challenges that these technologies are trying to address? And one of the reasons I wanted to begin by asking you this question is it seems that it's certainly within the last 12 months,

[00:02:54] they seem to be a renewed focus on value add and ROI of any new technology now. So what are you seeing here? No, I think, you know, the shifts are becoming much more faster now. So what we discussed six, seven months back, certainly there is a lot more acceleration to the trends. Broadly, I see our clients trying to do the following three things.

[00:03:19] And then I'll relate that to the technology shifts that are happening under those. You know, number one, there is this resurgence of value based care. That's been a term being talked about for many years, but I think there's a lot of resurgence in the last few months. Second, personalization and the building of on demand care models. Technologies allow that now.

[00:03:46] Previously, it was more of a science fiction. And to enable both of these, you need data convergence. Data in health care especially has been in silos. So we need to converge that data to make sense out of it. So with these three broad health care themes, the trends where our clients are spending a lot of their time and energy and money, of course,

[00:04:14] number one being Gen AI. Of course, it was Gen AI and now it is progressively becoming a Gen TKI, but that whole theme of AI, Gen AI, Gen TKI. Second, because I talked about data and data in silos, there is this big theme of creating value from the data and creating actionable intelligence out of this data. And that's another area of focus.

[00:04:44] And last but not the least, especially in health care, is cybersecurity. This is potentially the target industry. Obviously, last year, approximately, the cost of each breach was about $10 million. So it's very expensive to have a breach and the data is sitting in silos and potentially some archaic systems.

[00:05:10] So this is another big area of focus for us. So these are the three technology focus area underpinning the three broad teams that our clients are going after. And to choose one of those that you mentioned, a Gen TKI, one of the biggest buzzwords of the year so far, and it means different things to different industries.

[00:05:31] So I'm curious, looking at health care, how would you describe a Gen TKI and how do you expect it to impact the entire health care market in 2025 and indeed beyond? Yeah, no, very good question. This is obviously the buzzword. So let's first just define what Gen TKI means. I'm sure most of your listeners would now understand.

[00:05:54] It is a system designed to pursue complex goals, but do that autonomously. And in doing that, make decisions, plan actions, and even adapt to changing conditions and environment.

[00:06:13] So this is obviously a big shift from the old days of passive data analysis to now being very active in data utilization so that autonomous decisions can be made, data can be gathered, and actions can be taken on that data. So it's a big shift. And even in health care, we do see agent TKI being prevalent.

[00:06:38] So I think initially, obviously, it will be used mostly to drive efficiency. As you know, health care system has been under pressure. It is actually a three-pronged pressure. There is a pressure of cost. There's a pressure of increased need for efficiency.

[00:07:05] And obviously, consumers who are used to what other industries have given them are also demanding more. So with all of that, I think agent TKI will be used to drive efficiency. It will simplify the more complex workflows. It will be used in decision support more with, obviously, the intervention of human in the loop, given health care is such an important industry.

[00:07:34] And accuracy is obviously much more important and critical here. So when we're looking at the impact of agent TKI on health care, how can we ensure greater transparency in these fully automated decision-making processes that exist within these agent TKI systems? Because again, incredibly important in the health care industry, isn't it? Yeah, absolutely.

[00:08:00] So I think in health care more than anywhere, it's very important to integrate human being into the reviews for accountability, for decision validation, especially in the critical processes like diagnostics and patient prioritization. So, you know, in those where medical decisioning is important or required, human intervention is important.

[00:08:27] Second, I think also maybe more creation of a task-driven AI versus a goal-driven AI that will allow, you know, more deterministic systems. And it will also make decision-making more predictable and auditable. So that's another trend, I think, what we are seeing and what we are talking to our clients about.

[00:08:51] And obviously, given the nature of this industry, the AI decisions need to be logged, need to be auditable. They need to be reviewable for compliance to health care standards. And that's obviously non-negotiable. And of course, when adapting to the constantly evolving AI regulations and ethical considerations, especially in health care, is crucial and equally challenging.

[00:09:18] So when implementing agentic AI solutions, what kind of factors should be considered? Because it is a buzzword. There is a lot of hype around it, but it's equally important that we do everything in the right way. So what should businesses be thinking about? Yeah, no, this is very important because the government and obviously the health care organizations are still defining AI-specific regulations.

[00:09:43] So as technology is evolving, regulation is catching up to the technology in many cases. So it's very important to continue to look at the guidelines that are already in existence, because those are the ones being enhanced, whether it's HIPAA, GDPR or FDA's AI-ML framework. So it's very important that the systems adapt and follow those guidelines.

[00:10:18] So the system is also important to ensure that the system is designed to avoid bias, ensure fairness, and prevent any unintended consequences, especially in the areas like diagnostics and patient prioritization, as we discussed. And it's very important. And I've seen that our clients are not just doing AI for productivity or cost-saving.

[00:10:43] Hence, it's very, very important to balance automation with human oversight in our industry. So I think that we should not take shortcuts. And I am seeing our clients doing a lot of proof of concepts, a lot of validation before they put systems into production.

[00:11:04] And right now I'm seeing almost 80% of the POCs, even when they are successful, not being put into production if the right governance is not there. And only 20% of those are actually being put into production.

[00:11:21] And I suspect for many people listening, they would have seen many stories on their news feeds around agentic AI, but not too sure how it might work in their industry or understand what it could mean in terms of impact and measurable differences it can make within their industry. So just to bring to life what we're talking about here, are you able to share some of the biggest use cases for agentic AI, especially in the healthcare industry?

[00:11:46] Just to bring to life what we're talking about here and the scale of the transformation we're talking about. Absolutely. Absolutely. Especially since we're talking of agentic AI and not just the broader gen AI, I think most of the use cases that are going live are more around patient support or in some cases clinical support, but with a lot of human intervention.

[00:12:13] So let me give you an example of a patient support use case. And this, you know, because many of us have been patients would understand a virtual agent for medication management and adherence, especially in long term illnesses, medication management and adherence is very, very critical part of, you know, our wellbeing.

[00:12:40] And the agent, the genetic agent is able to clarify prescription because many cases, especially older patients, it's a big issue, clarify dosages, even talk about side effects and be very empathetic in the responses. And can also be enhanced further to allow for it to work with other agents to schedule appointments.

[00:13:07] And if you talk to, you know, people who are in long term care diseases, this is of critical importance. And obviously, with the rise in healthcare cost, it is very hard for providers to provide these supports without building these agents who can be there with the patients at every moment in time. I love that such a great example.

[00:13:32] If we zoom out and look at AI as a whole overall, how do you see this technology enabling maybe further innovation in the payer, provider and life science markets, especially in terms of things like personalized data and advancing efficiency? Because it feels like there's a lot of opportunities in this area. Now, tremendous opportunity. I already talked about providers where obviously automation of clinical workflows and enhancing patient engagement is critical area.

[00:14:03] But among peers, you know, obviously they are under tremendous cost pressure. So they're looking at largely operational efficiency. So either it is claims processing or fraud detection. They are very critical areas to deliver operational efficiency for them.

[00:14:19] And then life sciences, you know, GNI is a big, big theme there, whether it is drug discovery or optimization of clinical trials or even improving regulatory responses or writing regulatory compliance notes. It's a big, big theme for life sciences market. So these are some of the areas in the three industries that we address, payer, treudas, life sciences and including medtech.

[00:14:47] Like, for example, you know, there are obviously what you call connected patients or connected monitoring rooms where data is coming from all of those devices and can be put together to a control room for managing on a real time basis. That's another use case that we are seeing in the medtech industry.

[00:15:09] And we have talked a lot about the technology today, and I'd love to bring it back for a moment to the bring it back to the patient impact here. That's where the magic happens. So how can organizations better channel things like advanced analytics and interoperability to ultimately improve those patient outcomes that are so important and address value based care? Again, no matter where people are listening in the world, these are huge factors, right?

[00:15:36] No, totally with you. Technology is just a way to deliver a better patient outcome and addressing value based care is a primary theme. You know, for a very, very long time in the healthcare industry, we have looked at healthcare as episodic. So issues happen and response gets given.

[00:15:59] What says looking at is a continuum that can bring, you know, lifestyle, behavior, social, biological, and other family history related genetic makeup together to determine the well-being and state of health of an individual.

[00:16:16] We now have ways and means to collect this data from technologies, you know, including variables or health management apps that each one of us have access to, and to build a comprehensive 360 degree view of patients. And that is very, very core, you know, as you look at integration and interoperability of healthcare data.

[00:16:40] So in the past, it was not possible either to get this data or to store this data and almost impossible to analyze this data without use of JNI. So all of those things are now possible. So I feel like we are the closest we have ever been to us seriously looking at value based care and models.

[00:17:03] And to deliver this capability right to the doctors or physicians at the point of care, hence augmenting their decision making. So I think this is the best outcome I can see from all the technology that is available today. And to address some of the concerns from patients and healthcare leaders alike around how data is being used.

[00:17:30] Any tips you can offer on what trust building frameworks are ultimately essential for ensuring that agentic AI is, yes, reliable, but also auditable? Yeah, I mean, obviously, the number one theme is explainability.

[00:17:46] Yeah, that is the single most important thing in the industry, like healthcare, then obviously implementing these models requires very clear documentations, the decision trees, and very transparent governance to ensure that when regulators ask certain questions, we have the ability to answer. And to achieve that, we have the ability to provide those answers.

[00:18:12] And to achieve that, we have to work from the very start at the design level to mitigate the potential concerns in healthcare. So data input validation is huge here. And standards like NIST or ENSI are very, very important to be followed.

[00:18:35] Yeah, I think as the example of things happen, we need to get a look over to the PSC over the years on. And that would probably occur. So each question gives us a look over to the things, we have a look over to the five things. I think that we can have a look over to the right and the core things, but a look over to the five things that we can do, we have a look over to the type of data in the data.

[00:19:03] behind certain decisions. So all of these are new themes that need to be followed to ensure trust building in frameworks for agentic AI. And as the overcautious ex-IT guy here, when we're talking about vast amounts of patient data processed by technology like agentic AI, what would you say are the key risks related to data security and indeed privacy that maybe

[00:19:32] leaders need to be thinking about? Because I think it's very easy to get carried away with the technology without thinking about the implications and what they need to be doing from the outset to avoid problems later on. But anything else you'd like to share around this? Yeah, no, absolutely. You know, cyber security data protection are vast areas and you can have conversations for hours just

[00:19:54] on that topic. But let me talk a little bit about specific areas to protect with regards to agentic AI, you know, with regards to the autonomous decision making. Because AI systems can take actions without human intervention, you know, which means obviously potential security risks could occur without human

[00:20:18] oversight. So very, very important to ensure that decisions are controlled. You know, unauthorized actions can be taken by agents, again, without human in the loop. So hence, more task based versus outcome based systems to be built. And then, obviously, as part of the attacks, you know, third parties could look at

[00:20:44] an agency. And so, you know, we're not introducing malicious actors to manipulate agent TKI. So we need to ensure that misleading data is not added or specific, or we need to craft specific inputs to influence decision making for to ward off these harmful outcomes. You know, last but not the least, very important that sensitive information, which could

[00:21:11] expose if an agent does not have proper data access control needs to be anonymized because you know especially with large language models once the data is in there it's very very hard to control so very important to anonymize data and be super conscious of what is going into these large language models some of the things but you know this is a vast emerging area

[00:21:36] our clients are dipping their feet in i have never seen health care in the past you know health care industry has been blamed for being the last ones to adopt technology but i think this is changing now especially with both the cost pressures and the pressures from the patients and also because some of the reasons that have kept the health care industry back have been around data

[00:22:05] you know data not being standardized data being in silos and gen ai actually very much solves the very issue of data and so i feel that i'm very very optimistic about leverage of gen ai and agentic ai in health care with all the caution that we talked about and if we take every tech trend that you've seen and that you're monitoring and every client conversation that you've had what excites you

[00:22:35] about the road ahead what makes you want to jump out of bed in the morning and excites you for the future of health care and technology yeah you know for a company which provided tech services the single most exciting happening for us is that suddenly business is a lot more interested in technology the number of business conversations that we are having or i am having is by far the most

[00:23:01] we've ever had in the past technology was seen as something for technologies and business was not as keen second trend i'm seeing is that you know when many years back when ai came into being it was hoarded very much within the data scientist community or the cio was not deeply involved in it but with

[00:23:24] regards to agentic ai and gen ai it's very much the cio who's actually the one building or delivering while business is deeply involved so both of these trends put us very much in the co-driver seat or co-pilot seat along with our clients to influence business so i'm very very excited about that well it's been a pleasure chatting with you as always but and for anyone listening that wants to dig a little bit

[00:23:50] deeper on anything that we talked about today especially the work that you're doing at city as tech anyway you'd like to point everyone listening that just wants to find out a little bit more oh the best place to find out more about us is on our website and there is a link there for where you can ask for more feedback and input and we would love to get any any requests for information and address those well thank you so much again for coming on i will add all links to

[00:24:18] everything that you mentioned there just to make it nice and easy for everybody to find you and get in touch and carry this conversation on that we started today and i think before our conversation many people heard of agentic ai dominating their news feeds etc but for me one of the things that i love talking with you about is looking beyond that and talking about building trust in autonomous health care systems talking about data privacy data security complying with health care regulations and building

[00:24:47] those ai regulatory frameworks but and the magical or the magic that can happen and the opportunities that we can create and unlock and as a result improve patient outcomes that's what we're talking about here it's not even about the buzzword agentic ai or technology it is about improving patient outcomes and how you're doing that so thank you for shining a light on this critical topic today thank you neil really love talking to you and thank you for your time today i think it's clear that

[00:25:16] the future of health care lies in striking that right balance between autonomous ai systems and yes most importantly human oversight but together that's where the magic happens so a big thank you to my guests for highlighting how agentic ai can transform patient care by automating some of those complex processes and support clinical decisions and enhance efficiency and yes the real challenge does lie in

[00:25:42] building trust transparency explainability and rigorous compliance frameworks must form the backbone of these systems and i think we covered that today along with the importance of addressing data privacy and security risks especially when ai is processing vast amounts of sensitive health care data but as my guest said explainability and accountability are key not just for regulatory compliance but for maintaining

[00:26:07] patient trust in a increasingly digital health care ecosystem which is only going to continue to go that way so for health care organizations the message that i took away from today's conversation is the path forward is clear integrate ai thoughtfully prioritize patient outcomes and ensure that every technological advancement contributes to better better care delivery most exciting takeaway though is that we appear to be on the cusp of a

[00:26:35] health care revolution where technology and human empathy converge to redefine patient experiences in a more proactive than reactive approach to health care but what role but what role do you see ai playing in shaping health care outcomes in the years ahead how can leaders ensure that innovation doesn't come at the cost of trust and transparency you've heard from me you've heard to

[00:27:02] today's guest and i know you've got your unique set of knowledge experience and insights and i want to hear those and i want to get those out of your head if you've had enough of me talking into your head and ears today let's hear what you've got to say please email me tech blog writer outlook.com and linkedin x instagram just at neil c hughes but it's time for me to prepare for another guest tomorrow we're going to go beyond health care we're

[00:27:28] going to choose a completely different industry it's what we do here every day and hopefully i will speak with you all again then but that's it for today bye for now you