What happens to your AI budget when one business question becomes a series of data queries across several systems?
In this episode of Tech Talks Daily, I speak with Rohit Choudhary, founder and CEO of Acceldata, about the work taking place behind an AI response. An agent may need to find information, call several tools, and repeat parts of that process before it can return an answer. Each step can consume resources, and the person asking the question may have little visibility into the infrastructure activity it creates.
Rohit uses the example of looking for jewelry without knowing which vault contains it. The analogy helps explain why an AI system can spend time and compute finding the right source before it does anything useful with the information. We discuss his view that repeated queries and tool calls are putting pressure on enterprise infrastructure budgets, particularly where data is spread across older systems and cloud platforms.
His banking example brings the problem into focus. Information can pass from mainframes through multiple processing stages before reaching the applications people use. Copies may represent different stages of transformation, and the data that is suitable for one business purpose may be unsuitable for another. An agent needs to use the appropriate source, while the organization needs to understand the reliability and cost of that decision. The banks are not named, and the example should be understood as Rohit's account of the patterns he sees.
We also discuss why data quality cannot be separated from governance. Different business teams have different requirements, but AI services can draw on information that crosses those boundaries. Rohit argues for controls that operate throughout the data pipeline, including access restrictions, quality checks, and reconciliation. The business question is whether a policy actually changes what a running system can do.
One of his clearest principles is that an agent acting for a person should follow that person's permissions when working with enterprise data. We talk about how common standards can coexist with decisions made by individual teams about where data should live and which infrastructure best serves a particular use case. The conversation does not offer a universal answer about who should own every cost, but it does make the case for consistent controls across separate budgets and responsibilities.
Rohit also describes Acceldata's X-Lake platform and its approach to bringing intelligence to distributed data. Alongside the architecture discussion, I ask how leaders can distinguish productive AI activity from repeated processing and experiments that never become useful services. He expects a period of experimentation followed by a closer assessment of which applications provide lasting value. Faster application development can widen access to information, but it also makes it easier to build something whose ongoing cost has not been fully considered.
Finally, we consider the consequences of using unreliable data. Rohit's credit and drug-trial examples are hypothetical illustrations of how errors can influence decisions. They reinforce a practical question for anyone expanding an AI service. Can you explain the information it used, the permissions it followed, and the resources it consumed?
I would love to hear how your organization is approaching this, so please share your thoughts. Are you measuring the full data and infrastructure cost of an AI answer, or mainly the model bill?
[00:00:00] Once a year, I open up a handful of places for people who are serious about launching their own podcast, but have never actually gotten around to it. And over the last 11 years, I've helped entrepreneurs, business leaders and Fortune 500 companies all launch their own shows. And here is how it works. You record your conversation on Zoom, Teams or whatever works best for you.
[00:00:23] You send me the audio, an episode title and short description, and I will handle everything else. From setting up the podcast host and distribution to getting your show on Apple Podcasts, Spotify and the many, many other podcast platforms out there. I will commit to working alongside you to help you launch your first season of 12 episodes in the bag. So you can start 2027 by ticking a season of podcasts off your bucket list.
[00:00:52] Think of me as your not just personal podcast producer, but also your accountability partner to make sure that you actually deliver those 12 episodes into podcast land. Now, I only offer this once a year with limited places available. So if you want 2027 to be the year you finally launch that podcast, head over to techtalksnetwork.com. Send me a message, send me a DM or even an audio message. I'd love to help you make it happen.
[00:01:20] Why can a simple question to an AI assistant create a complicated bill? Well, behind that answer on your screen, an agent might be searching several systems, repeating queries and calling tools that all consume resources that it actually doesn't need to. And if the underlying data is unreliable, you could be paying for the wrong answer too. Well, my guest today is going to talk about the connection between AI costs, trusted data and operational controls.
[00:01:50] We're going to discuss why existing infrastructure can come under pressure, how responsibilities can become blurred across teams and what it means to give an agent appropriate access. You'll hear practical ways to think about the data supporting your AI services, but also a few questions to ask before those services continue to grow.
[00:02:11] So if your AI budget currently resembles a restaurant bill that nobody remembers ordering, hopefully today's conversation will give you a useful place to start asking what is driving that spending. And with that scene perfectly set, let me introduce you to my guest now. So thank you for joining me on the podcast today. Can you tell everyone listening a little about who you are and what you do? My name is Rohit Chautry. I'm the founder and CEO of Excel Data.
[00:02:40] Excel Data is the world's finest data and AI platform. We are the number one platform for the autonomous enterprise, which wants to bring intelligence to their data wherever it lives. Well, it's a pleasure to have you join me today. There's so much I want to talk with you about. There's so many big headlines around everything we're going to be talking about. And you're someone right in the heart of this space.
[00:03:06] So to kick off, I've got to ask, why is rapid AI adoption creating infrastructure overspend? We're hearing a lot around tokenomics, et cetera. Even at companies, though, with experienced cloud and data teams, it seems there's no escape to some of these costs. But what's happening here? So what's really happening is that AI is behaving like a child.
[00:03:29] And it needs to go and look for answers, the best answers from a variety of different sources, whether that data is present in one section or the other. Because, you know, AI does not know exactly where to go look for data.
[00:03:48] And, you know, it is being given vague tasks, which it then converts into, you know, SQL queries or, you know, extraction queries or batch data processing queries, which then has to be run on multiple different infrastructures. So imagine, you know, if you had like a knowledgeable person and you send them to go look for jewelry in a vault, they would exactly know which vault to go to. Unfortunately, in the case of AI, we're not giving them exactly the vault number.
[00:04:18] So they have to go and find whether, you know, the jewelry is inside a vault or is it inside a cupboard or is it inside a room? So it has to go discover the place and then go find that and then bring it back to the person who actually sent AI to go figure out where it was. That's basically called the agentic loop. And these are different tools that essentially AI has to call.
[00:04:42] So what ends up happening is that, you know, a straightforward query translates into multiple queries and multiple tool calls. And each of those tool calls are expensive because, you know, that has to be computed. And that essentially is causing massive amount of blow up in terms of, you know, the number of times the data systems are getting accessed. Do we expect that to come down at any point in time? Unfortunately, not.
[00:05:08] So you and I, for example, you know, if you're looking at your marketing analytics, you're probably looking at, you know, the whole screen five times a day. But the same, you know, tasks are being looked up by AI a few hundred times a day. And that's causing the blow up over here.
[00:05:28] And just to bring what we're talking about here today, especially that agentic loop that you've mentioned, are you able to share an example of a hidden data problem that may be made an AI workload more expensive or less reliable than the leaders expected when they began the process? You'd have to name any names, but do you have any stories that would just really hammer home that point? I think so. I think, you know, I can share stories which relate to probably Doc5 banks in the world. Wow.
[00:05:56] I think over a period of time, what has ended up happening is that multiple technologies get layered in these banks and data is lying all the way from mainframes to cloud, you know, Databricks, Snowflake, all kinds of technologies. You know, when you look back at the last five, six decades, all kinds of technologies have been layered in all kinds of leading organizations to serve their customers. Eventually, be compliant and take care of your money.
[00:06:25] However, what it did was that it has created a sprawl of technologies, which is very difficult to sort of manage. There are multiple copies of that data which exists from source system all the way into, you know, the consumption systems. Not all of that data is reliable because, you know, some of that data is in transit. Some of the data has to be, you know, understood, interpreted and transformed for it to be, you know, making sense from a business point of view.
[00:06:53] So, the agentic loops have to be very clear as to when they are going towards the source and when are they going towards the consumption layer, which is already transformed. And so, therefore, there will be multiple agentic loops which will go and access data from different places, which means that you need additional compute for your on-premise system. But you also need a lot of compute for your cloud systems, which is going to increase the total amount of consumption on the cloud.
[00:07:22] As it is, cloud was getting expensive. Cloud was getting a little less secure given, you know, the attack vector from AI. And therefore, you know, the cost is just blowing up. And another phrase I hear a lot is end-to-end visibility. But what does that mean when data moves through pipelines, platforms, models, and business applications that are ultimately all owned by so many different teams?
[00:07:48] Yeah, I mean, the decentralization of data has been a thing because, you know, consumer will have its own data pipeline. Compliance will have its own data pipeline. Supply chain is managing its own data sets. So, there are the different domains within an enterprise which is dealing with, you know, multiple data sets which are going to be used for different business purposes.
[00:08:09] The model that most of the enterprises, the Fortune 500, the Global 2000 have held, is that while the technologies have been largely the same for these different teams, the ownership of the domain, the quality management of that domain, the rules that apply, you know, the quality that is good enough from a regulatory standpoint is not the same as from a consumer standpoint.
[00:08:33] And therefore, you know, the domain-centric rules have to be applied over the pipeline which is processing all of this data. And therefore, this has to be brought under control. Now, all of this data which is now being used for inferencing, which is the next generation stuff, has to be accurate because AI is actually going to give you answers which are otherwise going to be inaccurate.
[00:08:58] And so, therefore, the reliability of this data pipeline was, you know, if there was a time when you wanted these data pipelines to be reliable, this is the time, you know, it has to be highest. And so, the demand that we are seeing for connectivity between these different islands of technology, different islands and domains of data sets, this has been, you know, unreal from an enterprise challenge perspective as of how important this has become for the enterprise.
[00:09:28] And I think it's worth highlighting today as well that traditional data lakes were never designed for this new generation of workloads that we're seeing now. And I'm curious, where are you seeing them struggle most under modern AI demand? What are you seeing there? I think scaling all of these systems, that has become really difficult because if you think of it, you know, you cannot see consumption-based pricing when, you know, AI is going to call the same systems 100 times that humans are accessing maybe 5 or 10 times a day.
[00:09:57] I don't think, you know, if your fundamental business is growing 10 times, you cannot carry the OPEX off, you know, the additional AI expense because, you know, there's so much of lookup of data. And therefore, what is going on in addition to privacy and security concerns is that enterprises are looking for highly performant, low-cost infrastructure, which means that a lot of data has to be computed at rapid speed, but it cannot be done under current costs.
[00:10:27] And so, therefore, you know, one of the things that we did, you know, a couple of years earlier was that we reimagined how the data infrastructure is going to be spread. And that, you know, brought us to bringing the general availability of the X-Lake platform this year, which is, you know, data has to be in multiple lakes. It's a true mesh architecture, the implementation of a mesh architecture.
[00:10:51] And instead of, you know, centralizing all the data to a single location, you actually decentralize all the data and bring intelligence to wherever your data lives. And for people listening, how should their organization maybe better distinguish productive AI compute from duplicated processing that we've talked about, idle capacity or failed experimentation? Any tips or advice for people listening there?
[00:11:17] Well, unfortunately, I think all enterprises will have to go through this process of a little bit of efficiency and then rationalization. But at the moment, the situation is such that if you're not experimenting rapidly, you know, you won't be able to rationalize rapidly. So most of the data leaders and most of the data teams that I'm talking to, they're openly experimenting.
[00:11:36] I think there will be a time where, you know, everybody will find their footing in terms of what AI is persistently useful versus what appears to be useful. You know, there's a phrase that I use, which is, you know, there's sometimes an illusion to building because, you know, build time is completely disrupted. You know, you and I can start writing in plain English and build the AI applications that everybody, you know, will find very useful.
[00:12:05] And it's great because, you know, we get access to data, we get access to insights rapidly. But at the same time, some of that may not necessarily be useful in the long term. So there's going to be rationalization. The second thing that I keep thinking about, you know, on that is that, you know, if you already know how to run your business, then, you know, you are not going to make arbitrary queries every morning.
[00:12:30] Once you have settled down on the KPIs that matter to you, then you're going to basically look for that data every single morning because your business doesn't change day to day. Sure, it has a trend on those metrics. So therefore, you know, you're looking at that. However, it doesn't mean that, you know, you're doing everyday exploration beyond a certain point. So I think, you know, rationalization will start at that point in time. There will be more users of data.
[00:12:57] I think it's going to get truly, truly, truly democratized. You know, the wait for teams to come up with dashboards and hand it over to business, to technology, to different teams. You know, that's going away. Business technology is going to be super important. IT will have to adopt to the style of the business. And it has now the capabilities by which they can support the business way better than how traditional IT was able to. So I think there's going to be a lot of goodness in that process.
[00:13:26] However, you know, as is the case with any technology that the interest levels are peaking. And then at some point in time, you know, there's going to be rationalization of what is useful versus what was great to explore. A hundred percent with you. And of course, later in that conversation, we have to get to governance at some point. And governance programs often focus on policies and catalogs, et cetera.
[00:13:49] And again, I'm curious from what you're seeing here, are there any operational signals that listeners need to know whether those controls actually work in practice? I think if controls needed to be implemented and not to be spoken about, I think this is the best time. I think, you know, for years, governance has been talked about as an annual exercise that somebody comes, beats you up with a stick. And then, you know, all of you produce a report which says I'm compliant.
[00:14:19] The reality is that, you know, governance has always been an afterthought, but it's no longer going to be sufficient in the age of AI. I mean, just look at the number of incidents that are going on at the moment from a leak perspective. So things like security, access control, privacy, it has never been more important because your tools have to be exactly locked down.
[00:14:41] The sandboxes have to be restricted from the access that it gets as it's generating code, doing things, you know, just as I was giving an example that, you know, you're going and looking for different vaults. And you don't even know whether you're permitted to open those vaults or not. And therefore, the vaults have to be secured in the most appropriate fashion. So I think, you know, security has to become the primary thing that everybody is looking for. At the same time, you know, you don't have an option to clean up your data only when it gets consumption ready.
[00:15:11] Your entire data pipeline has to be secured from an operational perspective. So things like access control, things like data quality, things like data reconciliation. If there was a time where this was important, this is the most important time. And therefore, you cannot actually rely on previous generation tools. You've got to basically be the first and foremost, your opinion as an enterprise and your focus as an enterprise has to be on getting controls operational.
[00:15:38] Do you need AI agents that you can trust? Well, with an AI data layer providing real-time connection within your data platforms, you can trust your agents to provide accurate solutions. So scale your business by trusting your agentic AI accurately getting the work done for you. Trust its capabilities with Denodo. And you can do that by simply visiting denodo.com to learn more.
[00:16:05] And I suspect if I was to ask this next question inside any organization, it will also almost immediately turn into that Spider-Man meme with everybody pointing at each other. But maybe we can settle this argument today. Who should own the cost and the reliability of all that data used by AI when infrastructure, data engineering, and model teams all have so many separate budgets? Where does it fall? I think it's a very interesting debate. We've seen all that IT basically centralizes.
[00:16:35] It's almost like, and then there's a big bang because teams don't feel that they are empowered. And therefore, the buying capability goes to the individual lines of business and their technology heads. And then there's a phase of consolidation and rationalization of technology. I think we're going through the rationalization phase because standards have to be changed. Everybody and, you know, there's not a possibility where you can start leaking data from the consumer unit.
[00:17:02] As an example, if you're inside a bank and then leave it for compliance to come and fix it for later. So I think some central standards have to be applied. And that has to be uniform across, you know, different business units. And therefore, the runtime becomes way more important.
[00:17:23] So, you know, when we were working on this concept, which says all your runtimes, whether it is for purposes of data or whether it is purposes for AI agentic loops, they will follow the same method and omission. For example, you know, Rohit's agents should behave exactly how Rohit would behave as he is interacting with the, you know, enterprise data.
[00:17:45] It requires a little bit of thought to have this consistent approach towards governance, access control, security, data quality and compute and the capacity to spin up compute across these various infrastructures. It is required. Otherwise, you know, it just ends up not working for the enterprise. So I think, you know, there has there have to be standards which will then have to be followed. The ownership of the infrastructure, what is the most appropriate infrastructure?
[00:18:11] I think, you know, the teams will still have to decide, you know, whether they should maintain an active copy or a passive copy or should they go back to all the way to the source. It would really depend upon their use case. So at this point in time, the only thing that can actually make this whole thing work is essentially the the most sincere application of the enterprise standards. So many great points there. And I'd love to leave people listening on maybe a word of caution here.
[00:18:37] What could happen or what are the consequences for both employees and customers if leaders scale an AI service before they can see whether its underlying data is trustworthy? We're back to moving fast and break things. But what would happen if if they do race ahead there? What are the consequences that they need to be aware of?
[00:18:57] I think, you know, from a very simple standpoint, if you were applying for a credit card and your credit numbers were not verified, then you are going to see outcomes which are not in the favor of the consumer. And there are legal liabilities that are regulatory challenges. All kinds of things that you hope would never happen in your business is happening. Consider drug trials.
[00:19:19] If you got, you know, incorrect information about the outcome of the new drug that you're testing and you've not verified, it just amplifies an incorrect result. I think, you know, if you draw a parallel between what's happening in terms of business outcomes, the simple way of thinking about it is that, you know, engineers who are good, they've become 10x better because of AI, which is amplifying their capabilities.
[00:19:45] And so every mistake that you make, every point that is wrong in your data gets amplified way more than, you know, it was earlier. So I think, you know, if you are not sort of taking care of the underlying infrastructure, if you're not taking care of basics, defaults are going to get way more amplified than they were getting amplified for humans.
[00:20:07] And of course, at Excel data, you provide an agentic data management platform that is able to simplify and speeds up the ability for enterprises to deliver trusted data for those AI and other initiatives, everything we're talking about here today. So for anyone listening who would like to carry on that conversation with you or your team or just find out more information about how you might be able to help them, where should they go?
[00:20:31] Well, the best place is to actually start exploring, you know, on our website, we put out some great content on LinkedIn, but the website would be exceldata.io. We've got some of the best use cases, case studies and resources. You know, when we started the agentic data management journey, we figured out that it's going to be a convergence of tools, technologies and personas. The language for the future is going to be English, you know, to interact with technology.
[00:20:57] You and I can build anything and everything if we have a structured thought process and we know exactly where we are going and the outcomes that we're pursuing. At the same time, it requires control over your runtime. And so it's a very, very interesting world because, you know, if you start finding faults in your application at 2 a.m., there is no way that you'll be able to control the outcomes, the consequences of having written a poorly designed AI application.
[00:21:26] So not only you need the controls that essentially provide high quality data to all your inferencing and post-training and fine tuning, but you also need a secure runtime which can run all your workloads with the same permissions so that, you know, Rohit's agents behave exactly like Rohit. So I think there are some great resources on our website to begin with and talk to us, you know, come have a conversation.
[00:21:51] Awesome. Well, I will add links to everything that you've mentioned and a few other areas. Go to techtalksnetwork.com. There'll be a blog post with everything on there. I'll try and embed a few videos as well just to further bring that to life. And we've covered a lot in a short amount of time today, so I want to encourage people listening to dig a little bit deeper, find out more information. But Rohit, a big, big thank you for taking the time to sit down with me today. Really appreciate your time today. Thanks, Neil. Thanks for having me.
[00:22:17] My big takeaway from today is follow the work behind the answer. An AI service can look straightforward to its user while creating repeated activity across systems with different owners, costs and permissions. Understanding that activity gives leaders a better basis for deciding what deserves continued investment. And his comments about operational governance, I think, also stayed with me because
[00:22:44] a policy needs to show up in how data is accessed, how it is checked and how it is used throughout the entire workflow. And before expanding an AI service, it seems sensible to ask whether the team can explain both the quality of its inputs and the cost of producing its results. Love to hear your thoughts, though. Let me know. Can your team trace an AI answer back to its data, back to its permissions and full cost of delivery? Maybe give that a try.
[00:23:14] What did you find? As always, let me know. TechTalksNetwork.com. Leave me an audio message or a DM, whatever it is. Connect with me on LinkedIn, Instagram, all the usual places. Let me know. But that is it for today. So thanks for joining me. And I'll speak with you all again real soon. Bye for now. Bye. Bye. Bye.