Building Trust in Production AI Agents With Caylent.
Tech Talks DailyNovember 02, 2026
3776
26:3620.23 MB

Building Trust in Production AI Agents With Caylent.

In this episode of Tech Talks Daily, I speak with Valerie Henderson, CEO of Caylent, about what has to happen between a successful demonstration and a system a business can depend on. The conversation centers on production readiness, including access controls, spending limits, traceability, workflow design, and the people responsible for the outcome.

Valerie describes why introducing agents requires changes to how work gets done. If an organization adds new technology without reconsidering the handoffs and decisions around it, the hoped-for productivity gains can remain difficult to achieve. Her advice is to begin with a bounded problem where automating a few steps could provide a clear benefit, then build on the evidence rather than starting with an ambitious network of agents.

We discuss the difference between read-only workloads and agents that can alter live systems. Valerie says many customers are already comfortable with agents retrieving information, while autonomous operations such as self-healing infrastructure are concentrated among the most advanced customers she sees. Those observations reflect her experience with Caylent engagements. They do not establish a universal level of enterprise adoption.

For me, one of the most useful questions is what meaningful human oversight looks like. Valerie is clear that people remain accountable for the instructions and access they give agents. She emphasizes deliberate testing, appropriate governance, and the ability to understand what a system has done. We also discuss how agents can repeat work without producing anything useful, creating a bill that needs to be considered alongside accuracy and reliability.

The conversation moves into Caylent's own operating model. Valerie describes moving away from assumptions built around human time and materials, making existing technical knowledge easier to reuse, and changing delivery incentives to reward speed and customer satisfaction. It is a practical reminder that organizations adopting AI may need to reconsider what they reward as well as which tools they buy.

She also shares a customer example involving Smarsh. According to Valerie, redesigning its software delivery process with agentic AI freed engineering capacity that could be reinvested in the business. She describes that capacity in financial terms, but the interview does not establish a measurement period or separate capacity value from realized cash savings. The useful lesson is the connection between a defined objective and a decision about how to use the resulting capacity.

We touch on Caylent's agentic AI readiness research, while keeping the broader discussion focused on practical experience. The useful question for leaders is what controls and evidence should come before greater autonomy, and how they will reassess that decision as the system changes.

Finally, Valerie explains how she thinks about success for employees as well as customers. She wants technologists to have time for difficult problems and opportunities to develop their skills. She also argues that customers should be able to explain the value of the work they have undertaken. Model performance is part of that assessment, alongside the environment, permissions, costs, and business process in which the agent operates.

I would love to hear how your team is making these decisions, so please share your thoughts. Which responsibilities are you ready to delegate to an AI agent, and what evidence would you need before going further?

Useful Links

[00:00:04] What would you need to see before allowing an AI agent to act inside your business? A convincing demo might show that it can complete a task. But production, that's something that brings questions about access, spending and accountability. And ultimately, what happens when something could go wrong? Well, today I'm joined by Valerie Henderson. She is the CEO of Caylent.

[00:00:32] And together, we're going to discuss how organisations are moving from AI assistance towards greater autonomy. And we will also talk about starting with bounded work, measuring useful outcomes and giving people a role in some of those decisions. And Valerie will also share how Caylent has changed its own delivery model and incentives as AI becomes part of its own software development.

[00:00:56] So for anyone facing any form of pressure to deploy agents quickly, today's conversation will offer a way to take a step back and think about readiness beyond model performance. Yeah, you don't need another impressive demo to appreciate the problem. But you do need to know what the system can do, what it should do and who's responsible for its actions. And that feels like the perfect moment to introduce you to Valerie right now.

[00:01:26] So thank you for joining me on the podcast today. First of all, can you tell everyone listening a little about who you are, what you do, especially after stepping into the role of CEO back in March? Tell me, tell everyone listening about you. Yes, Valerie Henderson, CEO of Caylent. I've actually been at Caylent for over five years now and I've had the pleasure of being here through a tremendous growth curve. But yes, I did just step into the CEO role earlier this year.

[00:01:56] It's funny, time is a construct, so I don't even think about it at this point. But Caylent is a really special organisation. We are a cloud-native AI professional services company that works really closely with our customers to bring their ideas to outcomes.

[00:02:12] And while that's the 10,000-foot view, we have an incredible amount of capabilities to help customers in a very noisy, overwhelming time and space right now actually drive ROI from this technology. And anytime there's been a major shift in technology, there's been a seam created between the tech platform and the business outcomes. And I believe the reason this time it's different is AI said, it's all going to be easy.

[00:02:41] It's just, it's AI. And we've seen that that's not the case because, number one, humans changing behaviour. You've got to change workflows, operations, all of that. It's a lot of change that's needed to get the most out of this technology. It really is. And time is just a construct. I'm going to be using that line, believe me. But Caylent time is just a different beast, for sure.

[00:03:05] Now, if we have a look at that, I think many companies are moving on from just asking questions to now asking agents to go and act on their behalf and complete tasks. So I've got to ask, from where you're sitting here, everything that you've seen over the last few years, what is changing for a business when software goes from just advising to actually acting on its own behalf? Yeah. Well, in short, everything. So I know that that can be really overwhelming for any customer to hear.

[00:03:35] And that's part of why Caylent exists, is to break that down and help customers understand what needs to change first. Obviously, for decades, work has run on a cycle of handoffs and sinks. And now things are happening concurrently and seamlessly. And humans get to have the most effective conversation, which means they can make decisions and quickly correcting what agents did.

[00:03:57] So we think about how we're helping customers do build and then actually just get into the point where they're mostly just making decisions really quickly as the human in the loop. We talk also a lot about the steam engine to the electric engine. And when the electric engine came out, they just dropped it in to where the steam engine was.

[00:04:20] They changed nothing about the factory floor, the different pieces, parts of that workflow, and they didn't get productivity for 30 plus years. So that's a huge part of what needs to happen. You have to redesign processes. So agents, human workers, and the legacy systems can coexist with a more clear view on the human in the loop and the change management. It's the operating system piece.

[00:04:46] And we did a survey, and it shows 60% almost of enterprise leaders are ready to let agents run autonomously in production, but only with defined guardrails. And we partner with AWS. We only build on AWS. It's the one cloud we build on.

[00:05:05] And that's part of why I think Bedrock has been such a powerful solution, just with those built-in guardrails and really getting people more security, more operational structure from the beginning. If we also take a look at that, I went to a few tech conferences recently, and one that struck me was last year. It was an industrial AI event, and they were talking about individual agents and working for individuals. This year, that event, they were talking about teams of agents.

[00:05:33] So in 12 months, it's gone from individual agents to absolutely teams of them there. So before a company deploys autonomous agents, from what you're seeing out there, what are the big decisions leaderships need to make and what it can access? And most importantly, who is accountable? That is a big question that many struggle to answer now as well. If one of those agents has an unexpected delivery item there, what's the best way around that? Where should they start to avoid mistakes later on?

[00:06:01] I think we've always told our customers to think about a problem that a couple of steps being automated would add a huge amount of value and start there. Let's start very simple. And then I think if you start simple, you can build on that. The teams of agents and how they're working together, I think it's very exciting. But to your point, the access they have is a huge part of them actually being able to be productive.

[00:06:27] And we find that in terms of accountability, there's a reason why there's a human in the loop. The humans are the ones that are telling these agents what to do and where to go and why to do it. And at the end of the day, we have to be held accountable. And I believe that you should assume nothing. You have to be very intentional about what you are doing and building and testing before you're moving anything into production. That hasn't changed. That's been the same for a very, very long time.

[00:06:56] And it doesn't change now. And the security piece, I think that this is such a very full time for companies in cyber and how they're thinking about evolving their platforms. That's one area I'm just very fascinated to see how that evolves because it's just going to continue to need a heightened level of inspection and visibility. And I always like to talk about the known unknowns. And right now, there are a lot of them.

[00:07:26] And it can be overwhelming to customers, which is why I would say start small. We often start in HR or go-to-market motions where there isn't necessarily going to be some major issue on these agents and the work that they're doing. And if they make some mistakes, which they will, that's okay before we're moving things into more business-critical applications and departments.

[00:07:49] And, of course, there will be lots of leaders listening to our conversation today or increasingly focused on ROI, measurable value and delivery, et cetera. Plenty of AI pilots. They look great in a demo. But what should a CEO listening, what should they be asking before believing that the agent is delivering that real repeatable business results or just delivering value? What should they be looking for?

[00:08:12] I think the things that they have to be looking for always are how are they organized up the stack so you can think about the infrastructure, the data, and the applications. And I often find that because of the pressure customers are getting from their boards, their customers, their employees to leverage this technology, you can want to just deliver something.

[00:08:39] And I think at the end of the day, if you are not thoughtful about what you have and what you can actually deliver securely, you're never going to get that momentum. And I actually feel it can be more dangerous if you rush these things out. And even if they look great in a demo, if you're not actually able to turn it on and show ROI over time. And my belief is it does not have to be a crazy amount of ROI in the beginning. We have some customers that maybe are saving $10,000, $20,000, $50,000.

[00:09:08] But the way these things compound over time can be so meaningful. And that's the other piece as well, just being thoughtful of, okay, where are your expectations? And I think starting in the bar low and rising it over time is the way to make sure that you're actually able to codify this within a business. Because otherwise, it will always feel like something extra. I mean, at Kalen, we had to reinvent the way we deliver.

[00:09:36] It was always human-based and based on time and materials. And we had to totally change what that looked like because AI changed the way our coders worked, our developers worked, how all of our actual IP could help power and speed things up across the rest of our projects. So I think, again, it's looking at the whole stack and then your operating system as a company.

[00:10:00] And if you don't have clarity on how this is going to fit in there, a demo is just literally a demo. A hundred percent with you. And if we look on our news feeds and maybe even go on Netflix, the hugging face and so many big headlines there has made a lot of people a little bit security aware, certainly more so than they were. So on the topic of deploying AI agents securely, I was reading that you've embedded AI deep into your AI-first approach.

[00:10:29] But tell me a little bit more about that and how you've been able to scale your services organization in this AI era. But most importantly, doing it securely. So that's, by the way, I mean, we think the most important thing from the beginning is security. And the great thing about AWS is obviously integrated in everything they've ever built is that notion. And so for us, that's just been part of our DNA.

[00:10:55] So we don't do anything without thinking about the governance, the access first. And the cool thing is we have an incredible group of technologists that work within this business. Now, they're spending time with our customers every single day helping build. But as people finish projects and have some time on our bench or our builders team, they were able to build us an incredibly powerful platform that we've been able to build a harness

[00:11:22] that has allowed us to scale as the technology has gotten better. And I feel like there was a period of time where there was this belief of just token maxing. And I think now we don't even think in the context of the tokenomics as much as the taskonomics and how we're thinking about what that looks like. And we exist to serve our customers. And right now, what our customers need is they need expertise and they need agility and speed.

[00:11:51] And so when we thought about how we are delivering, we thought about how can we help our customers in both of those categories. Now, the amazing brains of our technologists have a deep level of understanding of the different platforms across AWS and Anthropic. But the speed thing, the AI unlocked what that could mean. And I think there's some element of it that's table stakes. But for us, being able to look at how we're using it, package it,

[00:12:19] and actually build a wrapper that takes all of that IP and make sure that when people start a project, they have access to that and knowing what someone has used and be able to accelerate the outcome for customers. The other thing that we've changed is our incentive structure. We want our delivery organization to be incented to help customers deliver and get what they need faster and better,

[00:12:46] meaning a higher satisfaction, CSAT, and speed. And that has been an unlock for us in an incredible way. I mean, obviously, that changes the way people think about their behaviors, but it's been amazing because I don't think that that's traditionally how delivery organizations have worked. And that goes back to you have to rethink all things from first principles as you're integrating this technology into your company.

[00:13:13] And as we record this, we're literally weeks away now from 2027. And I think when we look back, 2026 will be remembered for agentic AI and the huge advances that have been made there. And many CEOs will be behind the curve looking for 2027 to make their move or try and improve what they're doing. As CEO, as someone that's having huge success here, I've got to ask you, what does agentic success look like to you? I would say agentic success looks like to me a couple of things.

[00:13:42] Number one, I think really healthy organizations have incredibly amazing talent and retention. I think one of the most important things for a company like Kalen is our ability to hire, develop, and retain our technologists. And for me, our technologists want the ability to spend their brainpower on the most important and critical and hardest problems.

[00:14:07] So if we give our organization the right agents, the right tooling, the right platforms to give them that time and that ability to be creative and work on those problems, that's success because that pulls through to all of our customers. And I think that that's the piece that is harder to measure on the ROI front. But we do look at actual retention from an employee standpoint as an ROI.

[00:14:33] Why? Because if a technologist or an architect is thinking about where they wanted to work, they want to go work somewhere where they're going to learn and they're going to be pushed and they're going to have access to the latest and greatest technology in this space. And that's Kalen. So we really think about agentic success from an internal standpoint, from that perspective. And then for our customers, if every single one of our customers can answer the question of, what's been the value of this? That is success.

[00:15:04] If they cannot answer that question, I would say that we have failed. Because you should not take on this work in building these agents within to these workflows unless you can answer that. Now, I understand some level of experimentation that needs to happen, but we have to do the work to be able to break down these problems and these organizations and these teams to be able to answer that. Love that. Absolutely perfect. And I've also been doing a little research before you joined me on the podcast today

[00:15:33] to learn more about some of your recent research. And there was a standout stat in there for leaders listening that nearly 60% of enterprise leaders, they're already running agents autonomously in production. So great news there. But most are still starting small and then expanding as agents continue to prove themselves, which is a sensible approach. But where do you think people still need that sign-off? And where is it maybe safer to just let agents act on their own today? I know it's a big move.

[00:16:01] It's almost like letting your kids leave home, right? What is the right moment? I think what we're seeing is a lot of our customers are already doing read-only workloads. And I think that is an area that there is a little bit more comfort in how agents are operating there. I think outside of that, in terms of true autonomous operations, so you can think about self-healing infrastructure. That's only happening at the most advanced customers.

[00:16:28] And frankly, within Kalen, our own managed service practice leverages that. But it requires very strong expertise, and to your point, security, governments, and the right platform. That is essential. I also feel like there is just a trade-off of autonomy versus risk. And I think that should always be re-evaluated. So thinking about the accuracy, the reliability, and the capability to improve.

[00:16:57] Can agents test things without side effects? And I feel like that removes a huge amount of the inherent risk. And the other piece is agents can iterate and do nothing useful and run up a bill, and that's super wasteful. But if you have the right workflows, the right guardrails, the right traceability, the right limits, which that's something that our Amazon Bedrock agent does in a really purpose-built way, you can be so successful here.

[00:17:27] And again, that is the stuff that is the hardest. It is not sexy. It is not flashy. It is really breaking down these workflows and the access, the platforms, and how you can make sure that you're doing this responsibly. Again, for people listening here, many companies will be moving from AI that recommends an action to agents that will go and take one. What changes are you seeing when software is given authority inside a live business process? What kind of things are you seeing?

[00:17:56] Any surprises, delight? What kind of stories are you hearing? Yeah, I feel like I am truly inspired by our customers because, like I said, this is not easy work. But I've been super impressed. We have a customer, Smarsh, that basically rebuilt their entire software delivery process with Agentic, SCLC on Amazon Bedrock and flawed code. And millions of dollars of engineering capacity were freed up.

[00:18:24] And they were able to reinvest that back into the business to continue to innovate. And to me, that's really amazing, being able to break things down and then build it back up. And they were really clear on what they were trying to achieve. And at the end of the day, with that return, they said, okay, we have an opportunity to jump ahead of the competition to continue to push ourselves to be better. And at Caylent, we're disciples of only the paranoid survive.

[00:18:52] And getting to work with customers that are thinking about things from that perspective, too, is amazing. And so I would say I go more into the delight rather than kind of the fear, uncertainty, and doubt side of where we are right now. I think in any given time with a shift like this, you have dystopian and utopian thoughts. I believe things are often somewhere in the middle. My mom used to say things are never as good as you think. They're never as bad as you think either.

[00:19:19] And I think right now that that's something that you have to really ground yourself in. And the cool thing is we're getting to work with customers that are seeing ROI, and especially in that software development space. And you can see that in a lot of the data. You have a ton of economists doing work on where this is actually showing up. And right now we work with a ton of technology customers, and that's a massive opportunity. And how you make that put it back into your business to go even further faster.

[00:19:48] And also when I was reading up on you guys, I think I read that there was a big obsession with model accuracy over the last couple of years. But I think I was reading that you were saying that model accuracy is no longer enough to judge whether an agent is actually ready to go into production. Can you just expand on that for me, for anyone that's not seen that? Well, I think model accuracy is just one small part.

[00:20:11] I think you have to think about there's so many different models, agents, and there has to be, in my opinion, a sense of experimentation. But you also want the right expertise, which goes back to why companies like Kalen and other organizations are so important right now. I feel like what we do has never been more critical than it is today. And number one, I think that there is a sense of, of course, excitement about that.

[00:20:40] But like I said, a deep level of responsibility is needed right now. And you have to think about things, like I said, in terms of the security, the guardrails, the spend, the access. And access, by the way, is one of the most challenging things. And you think about how simple it should be, but every customer that has worked with different platforms for single sign-in, we all know that that is not easy.

[00:21:07] And if it's not easy for humans, think about what that looks like for agents. So just thinking about those building blocks is really critical. So regardless of how good the agent is, if you are not, again, breaking these things down and being very smart about what and where and when and how, it doesn't matter. Yeah, completely agree. And I think we will have to get you back on early next year to talk about identity access management. It's a whole podcast episode on its own. And I was saying a moment ago, 2026 has been the year of agentic.

[00:21:37] If you can share what you've been up to this year. But most importantly, any teasers you can leave us with on what you're looking towards, what you're working towards next year? Anything you can share there? It's crazy to me that we're weeks away from 2027. I think this year has felt the shortest and the longest of any in my career. What I would say is we know that there is such a massive amount of pressure to adopt AI.

[00:22:04] And from my perspective, Caitlin being able to alleviate that pressure for customers is the most important thing that we do. And that means we need to see around corners. We need to be as far ahead as we can. I think being ahead of the frontier is not possible. But how do you take the frontier and bring the value to customers and help meet them where they are? And now is the time.

[00:22:29] And the other piece is how do we continue to drive the critical development of our people? And you've heard a lot about FDs in the last few weeks and months and all of the things. And there's a lot of noise around that. From my perspective, the expectation of an FD is AI fluency, a deep level of engineering expertise and true customer leadership. And there's no compression algorithm for experience.

[00:22:57] So just being able to make sure that we've got the right path for our people to be able to get into those roles is so, so important. Yeah, completely agree. And I think it's a powerful moment to end on. And for everybody listening, I will include a link to the research that I mentioned a moment ago so people can check that out. But anywhere else I should be sending them if they want to connect with you, your team, or we'll just find out more information about all things, Caitlin. Yeah, no, I love it. Thank you so much. This has been really fun.

[00:23:27] And any time, I'm happy to jump on with you. You're such a deeply curious person, which we love. You can find us. We have YouTube. We have LinkedIn. I would say LinkedIn has been really powerful for us. We've got some incredible customers and stories. And, of course, our website as well. So please, please come find us, Caylent.com. And, again, really looking forward to chatting with you again. Awesome. Well, for everybody listening, on techtalksnetwork.com, there will be a blog post associated with this episode.

[00:23:53] There will be videos, LinkedIn, everything we've talked about, including that research. I urge people to check that out. But as I said, the door is always open for you to come on and join. There's so much we could talk about. We could have spoke for another hour quite easily. But just more than anything, thank you for coming on and sharing your story. Really appreciate your time. Thank you so much, Neil. One of the many thoughts I'm going to be taking away from my conversation today is that trust in an AI agent depends on the environment around it.

[00:24:22] I'm talking about access, spending limits, traceability, and a clear purpose. All these things shape whether a useful demo becomes a dependable service. That's what we're aiming for here. I'd also appreciate her point about giving people room to work on those harder problems. And measuring that benefit could require looking at employee development and retention alongside the results that have been delivered to customers. I think that's something that's often missed.

[00:24:51] And for leaders deciding where to begin, one question is whether a small workflow has a clear outcome and consequences that the team can understand. And from there, autonomy can be reassessed as evidence grows. So a massive thank you to my guests for joining me.

[00:25:11] I will add everything that we've talked about today in the useful link section on the blog post associated with this episode over at techtalksnetwork.com, where you will find more than 4,000 different interviews. But I also want to hear from you. What evidence would persuade you to give an AI agent authority across your business? And what would remain yours? I'm sure there's a few stories out there. I want to hear them.

[00:25:38] Send me an audio message at techtalksnetwork.com or connect with me on LinkedIn, Instagram, X, all the usual places. You can find me at Neil C. Hughes. Nice and easy to find. Do you need AI agents that you can trust? Well, with an AI data layer providing real-time connection within your data platforms, you can trust your agents to provide accurate solutions.

[00:26:03] So scale your business by trusting your agentic AI accurately getting the work done for you. Trust its capabilities with Denodo. And you can do that by simply visiting denodo.com to learn more. But that is it for today. So I'll be back again tomorrow, waiting in your podcast feeds once again. But hopefully I'll get to speak with you again. Bye for now.