Why do businesses replace the AI model when the failure may have started somewhere else entirely?
In this episode of Tech Talks Daily, I speak with Richard Shaw, Technology General Manager for Databricks in the UK and Ireland. Richard leads the field engineering organization that works closely with customers on data and AI problems, giving him a practical view of what happens when promising agentic AI projects meet production workloads.
Richard argues that the model often receives the blame because it is the most visible part of the system. The actual fault may come from stale data, missing business context, inconsistent permissions, an unsuccessful tool call, or another point in the workflow. Replacing the model before tracing the request from start to finish can recreate the same problem in a new place. This is why lineage, end-to-end tracing, and continuous evaluation matter once an agent moves beyond a controlled pilot.
We discuss what a production-readiness rehearsal should include. Richard recommends realistic data, realistic user volumes, unauthorized requests, ambiguous questions, failed tool calls, and tests of what the agent should refuse to do. Teams also need agreed standards for quality, security, cost, and auditability, along with a clear decision about which actions an agent can complete independently and where a person must review or approve the result.
The conversation also looks at model choice and infrastructure cost. Richard believes the strongest test is performance on the organization's actual work rather than a benchmark leaderboard. A frontier model may suit complex reasoning, while a smaller or open-weight model may perform routine extraction or classification at a lower cost. Access policies, observability, and spend controls need to remain consistent as those model choices change.
Conversational analytics creates another governance challenge. Databricks customers such as Virgin Atlantic and Repsol are using natural-language tools to make company data easier for employees to question. Richard says wider access should preserve existing permissions, ownership, definitions, and lineage. An answer becomes far more useful when the user can see where it came from and which team owns the information behind it.
We also cover the boundary between historical analytical data and fast operational workloads. Richard describes how Databricks positions the lakehouse for broad enterprise context and Lakebase for immediate reads and writes, such as updating an account, placing an order, or storing agent memory, while keeping both connected to a common data and governance base.
Are companies ready to trace and test the whole AI workflow, or are too many treating the model as both the hero and the culprit? Listen to the episode and share your thoughts with me.
[00:00:03] Why does the AI model receive the blame whenever an enterprise AI agent fails? Well, my guest today is Richard Shaw, Technology General Manager for Databricks here in the UK and Ireland. And he will argue today that the real problem can appear anywhere. And it could come from stale data or weak permissions or missing contacts or even a failed tool call.
[00:00:30] So today I want to discuss what companies should be testing before moving an agent from a promising pilot to put it in front of thousands of users. And also discuss why model choice should follow the workload rather than the generic leaderboard. And how conversational analytics could also widen access without hiding ownership or lineage.
[00:00:56] And Richard's got loads of real world examples from Virgin Atlantic to Repsol. And talk about where employees are asking governed company data questions in plain English. So if your AI pilot works beautifully until reality turns up, hopefully this conversation will help you find out why. But enough for me. Let's get Richard onto the podcast now.
[00:01:24] So thank you for joining me on the podcast today. Can you tell everyone listening a little about who you are and what you do? Thank you, Neil. I'm Richard Shaw and I'm the tech GM for Databricks for UK and Ireland. I lead our field engineering organization. And my team are the team who work closest with our customers to help understand their problems. And then work out how we can, leveraging Databricks, solve a lot of those business problems. A lot of the problems that are challenging them around the data and AI space.
[00:01:52] Thank you for taking the time to sit down with me today. I'm at 15 tech conferences this year. Predictably, agentic AI and AI agents is all everyone's talking about. And as a result, companies are eager to launch agents. But many still struggle with fragmented data, inconsistent permissions, and there's a long list of problems there. But what foundation do you think has to be in place before an agent can be trusted in production? I think it's the big question that so many are asking right now.
[00:02:22] Fundamentally, it starts with trusted and governed data. An agent or AI in general is only going to be as reliable as the information and importantly, the context it can access. We also feel that the permissions need to follow the user. An agent should only be able to access the data and the tools that the person is authorized to use. And then there's the context. And this has become much more prevalent this year. It's not just about access to data. It's the context around that data and the context around the business processes.
[00:02:53] Unity Catalog, part of our platform, provides the consistent governance. Certainly for the data side, the permissions and the lineage. And Unity Gateway then gives organizations visibility and control across the models and the agents and the tools they use. And then continuous evaluation helps them make sure the quality and security hold up beyond the pilot stage. I'll give you a good example.
[00:03:18] One of our customers, Virgin Atlantic, uses Genie Spaces to give employees a talk to your data interface, allowing the business teams to access real-time insights and operational reports in plain English. This has turned internal policies and manuals into knowledge tools. And the Virgin staff can actually access this through other platforms such as Microsoft Teams. Incredibly cool.
[00:03:43] I think the good news as well is conversational analytics tools, they allow many employees to question company data. So how does that change governance when the person asking might not understand the underlying model or indeed its limits? What are you seeing here? Making the data easier to access shouldn't mean weakening the governance. And if anything, governance becomes more important when many people can interrogate their company data.
[00:04:08] With Genie part of our platform, employees can ask questions of their data in natural language, while the organization's existing access and governance policies continue to apply in the background. These users should also be able to understand where an answer came from and who owns that information, rather than simply being presented with an AI-generated response. Now, data teams still play a critical role in defining trusted sources and business logic, making sure that the answers remain accurate.
[00:04:37] Repsol, the Databricks customer, uses Genie to give more than 2,000 potential users natural language access to govern enterprise data, with the unique catalogue controlling access and tracking where the information came from. And operational AI applications are increasingly needing both analytical context and fast transactional data, though. So again, for people listening, how should teams decide where the lake house ends and an operational database begins?
[00:05:07] One school of thinking is that they shouldn't need to worry about it. And actually, that's what we built within the Databricks platform. Yeah. And being able to load and analyze both analytical data and operational data. For the business users, they shouldn't have a concern about it. And actually, the team supporting the platform and the data, they also shouldn't need to worry about,
[00:05:32] hey, do we have massive concerns around the shape of the data where we place the data? Ultimately, you want the platform to be handled both those types of data and access patterns seamlessly. But the team should begin with the workload. The lake house provides the historical data and the wider enterprise context that helps AI understand the business. When an application or agent needs to read and write information immediately, whether that's updating an account, placing an order, or storing agent memory,
[00:06:00] that's where an operational database comes in. The dividing line is very much based on latency and interaction. Analytical questions can run on the lake house, whilst actions that need an immediate response should be handled by the operational database. The lake base, which is Postgres on the lake house, brings those environments closer together, giving teams a serverless Postgres database connected to the same data and governance foundation as the lake house.
[00:06:27] This means that teams can use the right system for each workload without creating another disconnected data silo or moving information between complex pipelines. And of course, there are a lot of opportunities and a lot of excitement here, but there are increasing concerns around things like performance and infrastructure costs as well. So all these things are slowly becoming board-level concerns when it comes to AI.
[00:06:53] So what workload decisions have the greatest effect on both without sacrificing reliability and blowing the budget by Q2 of the year? One of the biggest decisions is simply using the right model for the job. The biggest or most expensive model isn't automatically the best choice for every task. Unity Gateway gives organizations visibility and control across different models so they can manage access, usage, and spend consistently. The same applies to context.
[00:07:22] Giving a model the right enterprise information is more efficient than continually sending it huge volumes of data it doesn't need. Ultimately, teams should optimize for the best reliable business outcome, not the biggest model or the most infrastructure. And I'll add a comment based on a podcast I was listening to this morning on the way into the office.
[00:07:43] It is highly recommended now that the businesses look at open weights models alongside the closed weights frontier models, certainly for the cost and the complexity aspect. And with a platform like Databricks and Unity Gateway, it provides our customers that element of control. We support a number of the open weights models.
[00:08:09] And as businesses become more comfortable in evaluating them and using them, then they'll see an opportunity to balance complexity and cost as a lot of these open weights models are being picked up by enterprises. Let's assume that somebody has got the agent in place. They're working fine. They're getting the ROI from them. But what happens when an enterprise agent fails? How often is the real cause of the model? What is it?
[00:08:36] Is it lineage, permissions, stale data, weak evaluation, a collection of those or something entirely different? What is usually the cause of agents failing then? It isn't universal, but the model often gets blamed because it's the most visible part of the system. In fact, the model is the new database. If you go back a couple of decades into what used to be the core problem behind many systems, in reality, the problem can sit anywhere in the chain. The wrong information was retrieved. The data was stale.
[00:09:05] Permissions got in the way or something failed in the wider workflow. That's why end-to-end tracing really matters. Teams need to be able to follow what happened from the original request through to the context, the model response tools used and the eventual action. Data lineage and continuous evaluation then help pinpointing where something has gone wrong and where the quality is changing over time. Changing the model before you've identified the root cause simply recreates the same problem somewhere else.
[00:09:32] And I'm curious for people listening here, I always try and give people actionable takeaways. Are there any readiness tests that a data leader listening could run this quarter before allowing a successful AI pilot to reach thousands of employees or customers? Any tests that you've seen effective here? What's incredibly effective is thinking beyond simply a POC, run more of a production readiness rehearsal.
[00:09:58] Test it with realistic data, but realistic numbers of users recreating the level of demand and the type of usage as well. Test what the agent shouldn't do as well as what it should, including unauthorized requests, testing the guardrails, ambiguous questions and failed talk calls. Also, set the bar for quality.
[00:10:23] Security, cost and auditability before you scale rather than deciding what's acceptable. So this is much more than just a does it work. And be clear about autonomy. What actions can the agent take itself? Where does a human need to review, approve or step in? And when it comes to open and proprietary models, which very often now sit beside one another in so many enterprise stacks,
[00:10:49] what should determine model choice before beyond benchmark scores and procurement preferences? Do you see any mistakes being made here? And what should determine that choice? Yeah, the best test of a model is how well it performs on your actual work, not where it sits on a generic leaderboard. That means looking at the quality alongside latency, the cost, the security and the consistency. Different jobs will also need different models. Complex reasoning might warrant a frontier model,
[00:11:18] while a smaller model may handle routine extraction or classification more efficiently. The really important thing is having the flexibility to use open and proprietary models side by side and choose what works best for each use case. This goes to my previous comment. A unity gateway provides a consistent layer of access, policies, observability, and the important cost control as those models change. So model choice doesn't become a model lock-in. And again, for people listening,
[00:11:48] how can data teams maybe give their business users faster access to AI supported analysis, while at the same time making ownership, definitions, and accountability obvious when two departments interpret the exact same metric completely different? Firstly, giving people faster access to data only works if there's a shared understanding of what that data actually means. Yeah. Critical metrics need a clear definition, an authoritative source, and someone who owns them.
[00:12:16] Now, if two teams legitimately define the same metric differently, that shouldn't be hidden. Users should be able to see which definition they're working with and why. The Gini one makes it easier for business users to access AI support analysis, while Unis Catalog helps preserve the ownership, permissions, and the lineage behind the answer. Now, AI can make information much easier to access, but it shouldn't make accountability harder to see. Well, at the very beginning of our conversation today,
[00:12:45] we were talking about the kind of problems that you're solving for your customers. I'm curious, if you put all the conversations that you're having with customers and clients and what you're hearing on the show floor at tech conferences, etc., are there any trends in the kind of things that are people coming and asking for help with? Is it all around a Gentic AI or something else? What is everyone coming to you for at the moment? A lot of the discussions are around absolutely a Gentic AI, and it's how to scale effectively within an enterprise. Yeah.
[00:13:15] There'll be lots of POCs, proof of concepts, lots of small point solutions. And those, in many cases, have been proven out. Now, the C-suite, the executives, want to scale that further across the business, but it isn't that easy to do from their perspective. You know, it doesn't mean just, hey, add more and we'll be successful. Thinking back to the description of how to do a successful rehearsal,
[00:13:42] you know, there are more considerations than simply, did it work and how do we train the users? The choice, the cost, the control, etc. These are crucial, important questions for any business to answer. And be able to do that at scale, that's something that we're helping our customers do. There are tons of business problems out there, which our customers have an understanding of how agents could help solve.
[00:14:11] And as the cost is reducing of certainly the models, the token cost behind the scenes, it's giving businesses more confidence in scaling more AI-based and agent-based solutions out there. Now, Databricks has come from an earlier mission statement of democratizing data and AI. And that still stands today. Now, we're at that point where a lot of our customers and enterprises see that the technology can solve these problems.
[00:14:40] So now it's giving them confidence and truly doing that at scale. And that democratization piece is crucially important. Putting the tools in the hands of the users, they understand the business problems. You know, it's not a central IT team who understands what a business unit may be doing or a lot of their business problems. It's those business users. So now it's quite a step change from business users simply working with data, using BI tools and making business decisions. Now looking at their processes and going,
[00:15:09] hey, we now have the tools in place where we don't need to be a data engineer or data scientist. We can build agents and AI solutions. And our business has confidence that as we're doing that, all the security, all the governance, all the control is in place. And that's quite an exciting place to be because those companies who have done their evaluation and proven out and become comfortable and confident are now moving ahead pretty quickly.
[00:15:39] Your agents aren't producing accurate answers because they don't have a complete semantic understanding of your data. And Denodo is solving this and solving it through semantic consistency. Through semantic consistency, your agents can start making accurate predictions in real time. So see what else Denodo can do by visiting denodo.com to learn more.
[00:16:07] As you said, it is an exciting place to be right now. It feels like we're still very early on in the journey that we're embarking on here, but we have got one eye on 2027 already, which seems crazy just saying that out loud. But some are working right in the heart of this space. This year's all been about a gente, KI and the next step. But what excites you about where we're going next and what we can expect, not just for the rest of this year, but next year and beyond that? Anything that makes you want to jump out of bed in the morning and particularly excites you?
[00:16:36] The open weights models, very interesting space right now. And it's really healthy to not have just a handful of players in any one market. Yeah. A lot of the open weights models like, like Kimi, like GLM, like Quen, they're starting to get the attention of enterprises who would traditionally have gone for the biggest players because they're safer. They're now looking at those and going,
[00:17:05] let's evaluate them and understand, can we access them through a trusted partner like Databricks or others where we don't have to be so concerned about the provenance of the model? We're more concerned about how it helps our businesses. That goes hand in hand with harnesses. And certainly as a product company, as a software company, we are very much at the forefront of using models
[00:17:33] and using harnesses to build our own product. A lot of the conversations I've also been having with our customers alongside scaling out agents is also how do we evolve our software development lifecycle in the agentic world. And part of that is focused on, hey, what's a good harness we now need to use? But then the harness also affords you the ability to then leverage different models under the hood and then start really evolving your software practices.
[00:18:04] So helping our enterprises come on that journey as well is very much hand in hand with the adoption of agents. It's a natural progression. Part of that is driven by where our own platform stands today. On Databricks, it's a four-stack platform today, which means you have custom business applications agents built on top of the platform. And a lot of our customers are now leveraging Databricks
[00:18:32] to refactor or to develop net new business applications with a custom application front end. And they're wanting to understand, hey, how do we use AI to help build those applications? And a lot of that is built into Databricks itself with tools like GenieCode. But also, hey, if we want a more traditional approach where we're using AI to write the code and we're testing that ourselves, what does that look like? So all of these are interlinked because it does absolutely call on agents,
[00:19:02] but it is a fundamental shift in how businesses are thinking about how they're building business applications. And it's moving away from, okay, it's probably going to take us six months and a lot of expense to refactor this old application. Therefore, let's not do it for the moment. Actually, they're now looking at some of those applications going, we could do that in a lot shorter period of time. We could get it up to date for what we need and we can build more going forwards.
[00:19:30] So there's a little explosion going on of customers going, we're not constrained by traditional software development practices anymore. And it's not just one shot and ask the AI to do it. We can evolve our software development life cycle in a meaningful way, but also we can be that much more productive and competitive as well. So general agents for improving business processes, productivity, absolutely.
[00:19:59] Use of open weights models and frontier models, very exciting. And then harnesses. And then also then how businesses build using AI. So there's quite a lot to be excited about going forwards. There really is. And I think that is a thought-provoking moment to end on today. But for anybody listening there that would like to carry on this conversation, where is the best place for them to find you or your team online and ultimately find out more about anything we discussed and more?
[00:20:28] Where should I be pointing them? First and foremost, the Databricks website, databricks.com, and our blogs are the best starting points. We publish a lot of information, a lot of articles. There's also dedicated pages for the products we talked about today, including Genie Code, Unity Gateway and Lakebase. And you can find us on LinkedIn and YouTube for demos, event sessions and discussions with some of the teams. Awesome. Well, I will add links to everything that you mentioned there.
[00:20:57] So for anyone listening, just go to techtalksnetwork.com. Have a look on the blog post associated with this episode. I'll include links for everything, make it nice and easy for people to find. But as you said, it is exciting right now. And there's so much more to come. So it'd be great to get you back on next year, see how things are progressing. But more than anything, thank you for sharing your perspective today. Really appreciate your time. Thank you, Neil. Richard left us with so much gold there, didn't he? I mean, from when an agent produces the wrong result
[00:21:25] and why you should follow the entire path before replacing the model and check the request, data, context, permissions, response, tools, and final action. Otherwise, that same fault may simply appear behind a different model name later. And I think his production readiness advice is also incredibly useful too. Test with realistic data and user demand. Include unauthorized and ambiguous requests.
[00:21:54] And only then decide where people must approve an action. And agree acceptable levels for quality, security, cost, auditability before expanding access. Yeah, we're talking about moving that dial slowly up again. It's an analogy I'm hearing more and more. So a big thank you to Richard for joining us today. Remember, you can find Databricks, its product information, and its research at databricks.com. But over to you, when your AI agent fails,
[00:22:25] does your team investigate the entire workflow or just reach for a different model first? Love to hear your thoughts. TechTalksNetwork.com. If you scroll up to events at the top, you'll find out where you can meet me. I've got a lot of tech conferences coming up over the next few months. It'd be great to meet some of you in IRL, as the kids would say. But enough for me. I'll be back again soon with another guest. Bye for now.