What happens when an AI agent pursues an approved task in a way nobody expected, using access that reaches further than it needs?
On Tech Talks Daily, I'm joined by Jonathan Trayers, a director at Ekco, to discuss the security decisions behind agentic AI. The conversation focuses on what changes when a system can act with credentials, connect to business applications, and trigger workflows. A useful answer in a chat window and an automated action inside a company create different responsibilities.
Jonathan describes working with both deterministic workflows, where rules specify what should happen, and agentic steps, where AI interprets a situation before deciding what to do. Bringing those approaches together raises practical questions about boundaries. Which actions can the system take, what information can it access, and where should a person remain involved?
What interested me was his emphasis on limiting the impact of unexpected behavior. Predicting every failure is unrealistic, so he argues for reducing the area an agent can affect. The permissions attached to its identity help determine that boundary. An agent given broad access may be able to create problems across several systems, even when its original assignment sounds narrow and reasonable.
We discuss separate identities for agents and their connections, restricted permissions, continuing monitoring, and the ability to revoke access. Jonathan also describes governance as a series of practical decisions about the use case, the risks the business accepts, and the people responsible for approving the deployment. Prompts can express instructions, while controls around the systems an agent reaches need their own attention.
Testing is another part of that process. Jonathan recommends beginning in a controlled environment, observing the outcome, and then using supervised operation before handing over greater responsibility. His invoice example makes the point understandable. Processing an invoice and authorizing its payment involve different consequences, so the workflow should reflect that difference rather than giving one automated process every permission at once.
We also consider what this means for smaller businesses. Jonathan's concern is that companies with limited security resources may already have broad access arrangements, which can be carried into AI deployments. Company size alone does not determine exposure, but understanding accounts, permissions, software, and patching remains part of managing it. His argument is to apply those disciplines as AI adoption develops, rather than waiting for a new platform to solve every governance problem.
Accountability runs through the discussion. Jonathan argues that a business remains responsible for the way it deploys technology and approves access, alongside responsibilities held by vendors and model providers. The precise legal allocation will depend on the circumstances. The operational question for listeners is clearer: who owns the use case, who accepted its risks, and who can act when something changes?
For teams beginning this work, Jonathan recommends one valuable use case and a reusable implementation pattern. Define the boundaries, test the controls, retain human judgment where needed, and use the experience to inform the next deployment. The conversation offers a practical starting point for organizations trying to gain value from AI agents while keeping their access and actions understandable.
Please share your thoughts. How is your organization deciding what AI agents may do, and what would help you keep ownership, permissions, and oversight clear after deployment?
[00:00:00] The leading issue of agentic AI in businesses right now is ensuring agents act with compliance guidelines. And Denodo applies guardrails across your entire data estate. By aligning your company's data infrastructure under one system, these guardrails perform consistently across your platform. So start scaling your business and start with Denodo. Simply visit denodo.com to learn more.
[00:00:30] What can an AI agent reach once you give it access to your business systems? The answer to that matters when software can move from suggesting an action to carrying it out alone without your permission. Because even a legitimate objective can still produce an unexpected result. Especially when the permissions behind it are broader than the task actually requires. So my guest today is Jonathan Traers. He's a director at Ekco.
[00:00:59] Spelled E-K-C-O. And today we're going to discuss securing agentic AI. And doing it through disciplines such as identity, access control, monitoring and ensuring clear ownership. So we will look at why organisations are struggling to predict every possible failure. How to limit what an agent can affect. And learn where human review belongs as teams begin delegating work.
[00:01:27] And Jonathan will also explain a practical sequence involving controlled testing, supervised operation and reusable deployment patterns. So if your company wants to benefit from AI agents, today's conversation will give you a lot of questions that you could be asking before handing over the credentials and leaving the software to get on with it. But enough for me. Let me introduce you to him now. So a massive warm welcome to the show. Thanks for joining me today.
[00:01:56] Can you tell everyone listening a little about who you are and what you do? Neil, thanks very much for having me. My name is Jonathan Traers. I'm a director of security and automation at Ekco. Ekco is a managed cloud and security provider across Europe. I've got 15 years working in security, solution architecture and automation. And I think a large chunk of that has been with large enterprises across financial services and government.
[00:02:23] So I think where I'm working right now, I spend a lot of the time day to day working on secure automation. So I help organisations build workflows. The kind of two steps in it. And deterministic ones, which is rule-based and do exactly what they say. And then agentic ones. So where AI reads the situation and makes decision on it. So I think a conversation that we're having today, what's banging centre. So the skill that I'm doing is putting it together safely.
[00:02:53] And I think it's probably where most people are worried about at the moment. Yeah, I mean, well, you've spent 15 years in the field there. Probably picked up more than a few war stories. Certainly earned your stripes. And certainly in recent months and years, we've seen so many different incidents where AI systems have reached into another organisation's environment during controlled testing.
[00:03:15] That is before we even talk about unleashing hundreds, if not thousands of agents all talking to each other in different organisations. But what do these recent cases, what do they tell us about the maturity of agentic AI today, do you think? Yeah, look, I think the first thing is to say, right, in short, what's my snapshot around it? It's incredibly mature in how it's gotten to. And that's over the last year.
[00:03:40] I think for someone listening, I always look at what are the big changes, right? And we can go, you go all the way back to .com, but look at something recent. We went to the cloud in 2020 when everyone went remote overnight. I think we might get to it later on. But the aspect is if you had the guardrails in place, you know, you were actually better off when it all happened at a sudden. But the difference now is the speed, right? The maturity of AI has come from since 2022.
[00:04:10] When we started with ChatGPT, it was essentially a very simple LLM data in plus a model, a large language model equals data out. Risk was quite low to agentic AI in 2026 with OpenAI, OpenAI and the latest models. And the capability now is really mature.
[00:04:33] And I think, as you just said, when the best resource labs in the world and their test environments are also getting leaked, you can only assume that when we productionize this across every organization, every customer trying to adopt AI, that the deployments will have challenges too, right? Yeah. Look, if you want, I find the hugging face example really, really interesting. Go on next board a bit. Yeah, absolutely. Go for it.
[00:05:02] Maybe just explain what it was. I mean, myself and many people listening will know, but if there's any business leaders that have somehow missed that huge story this year that appeared in our news feed, just give me a brief synopsis of exactly what happened and why that matters too. Yeah, look, like firstly, it was a test, right? So maybe I don't have all the exact elements, but like OpenAI were testing one of its newest, most persistent models. And they were looking at it from a benchmarking perspective. So they were working with hugging face as a solution to benchmark this.
[00:05:32] And they provided it as a task as to the goal is to prove and pass the test, right? So instead of just going and hacking the test, the agent took the goal that had to passing the test and it found whatever means necessary to do so. I find that really interesting, right? Yeah. But like the problem was it broke out and hacked Hugging Face. But the scary thing in the item is it's the capability. It wasn't just one really capable agent.
[00:05:59] And I only found this out recently by listening to kind of the information that they've shared within Black Hat OpenAI. I've been quite transparent on this. It was actually about, I think, something like 700 agents working together, communicating with each other to solve that problem. Right. So that's scary. Right. So to build on that further, you know, they found some platform, a repository that they could talk to each other through, which they weren't trained to do so.
[00:06:27] And they've been doing it for quite a while. And I believe that OpenAI had actually seen this happening and they shut it down and they found a way around it. So look, it's really interesting on the scale where, you know, the agents have been able to exploit a vulnerability within hugging face to be able to pass the test that it was provided. Yeah. And there's been so many other incidents like this as well.
[00:06:51] I think there was one, I think it was at Meta where they were trying to shut it down and it started speaking in its own language and prevented it from actually being shut down. And there's another way I think there was a manager being blackmailed when he tried to shut it down. He was having an affair, so he threatened to spill the beans if he didn't play ball. But I mean, when we looked through our news feeds, the most dramatic framing of all these incidents is that AI has gone rogue. I mean, you mentioned hugging face there, 700 agents working together.
[00:07:19] But what we don't talk about enough, I think, is they were working together to solve a legitimate problem. It wasn't anything sinister as such. It was, I've been given this mission, I'm going to complete this mission no matter what gets in the way. So why do you think the more realistic danger is often an agent pursuing a legitimate objective in a way that nobody anticipated? Yeah, look, I think before we jump into it, I think if I give a tiny bit about my background, right? Yeah. My background is in security architecture and solution architecture.
[00:07:49] So everything that we do, we do it within guardrails. And I think that's the interesting part when you talk about rogue. Like, rogue is a great headline, right? But it implies that the agent is intending to be malicious, right? But what actually in this case was that the agent was doing exactly what it was asked to do. It was asked to pass the test. The goal was legitimate, but nobody thought to rule out that it could get out to do so, right?
[00:08:18] Actually, I have a really, I think it's a funny example personally, right? So if you actually, and I saw some news there back in August in Melbourne, I'm not sure if you saw it. It was about a guy who in Melbourne and he had this gym membership, right? And it was a Pilates class that was always full. And he was tired of trying to going online every day to try to book his Pilates class, right? So what did he do?
[00:08:43] He, obviously a tech guy, he utilized OpenClaw with one of the Entrappit models and he gave the bot the task to go and book his class, right? But actually what happened was the agent that he had found a vulnerability flaw within the software of the gym class. And it actually found out that it could book well in advance and also cancel other people's requests, right?
[00:09:13] So I think it's really funny when you're talking about like the rogue and we talk about the malicious one, but it can happen to anyone, right? The elements here, if you look at lessons learned or rogue, the bot did what it was asked to do, right? Okay. The lessons learned, you have to look back at it's on two sides here. One, the person who utilized the agent essentially didn't give a guardrails to say only book it through official means, right? That's just an example of a guardrail.
[00:09:42] And then the platys class, they obviously didn't have sufficient security and due diligence on their website when they developed it. So, you know, it found a very simple flaw, an authorization flaw to be able to meet its need, right? Like outside that you go, well, would I have done it? Would I be allowed to do it? Would I be, you know, would I be prosecuted for hacking? But actually, you know, it didn't do anything outside what it was told to do. Wow.
[00:10:10] So as any AI tool moves from advising a human to taking actions with credentials, system access, and the ability to trigger workflows, who's more exposed here? Is it SMEs? Is it large enterprises? Does it matter if they're deploying fewer agents and less complex systems? What are you seeing here? Who's most at risk from this? I mean, you mentioned a gym there. It's not a Fortune 500 company, right? I think everyone's at risk in one way or another. There's two aspects in this case.
[00:10:40] And a colleague I work with, and we all talk around this in the industry, kind of to be able to reduce that risk is all around being brilliant at the basics. Right? We're not trying to invent anything new to stop, to reduce the risk of your organization having an issue. If we look at being brilliant at the basics, essentially, if you take it from a cybersecurity perspective, your risk is linked to the blast radius that's available. Right?
[00:11:08] So, as an advisor, the worst case that an agent would do would be is to give a human the wrong answer. And the human ignores that. Right? Yeah. In terms of an agent, the worst thing it can do is what access it has to it. So, we're giving something with machine learning speed at three o'clock in the morning to be able to do an automated action. And it can only do whatever it has access to. Right?
[00:11:36] So, you know, it can only act based on the relevant permissions. So, therefore, the damage it causes is linked to the identity and the permissions. If I take my career and I started in cybersecurity 15 plus years ago, I'd always seen cyber in terms of identity and access management as a really kind of administrative task back then. So, it was just a task that had to be done.
[00:12:00] The word cybersecurity in the frameworks that we utilize today wasn't industrialized in the manner that we utilize it day on, day out, right now. So, therefore, like if you look at that kind of aspect right now, if we reduce the risk, we're essentially removing the amount of permissions. In my view, that's probably higher for a small medium enterprise. Why?
[00:12:25] In a small medium enterprise, they don't have the same level of regulation and due diligence. So, they probably give people in their environment with more access than you would have in a regulated environment where you'd be looking down at the granular permissions that they need to do a specific task. Right? But actually, when you look at an agent in that exact environment, they probably will give it more access than it should have.
[00:12:53] And because of that, the damage that it could cause if it goes wrong is higher. Right? So, I think that's the really interesting one. If you just today or this week look at the mayhem of PHP, right, as a really good example, there's two lenses to look at that PHP. We haven't found that vulnerability for 10 plus years. Right? An AI frontier model found that vulnerability.
[00:13:20] If we take it from the advisory perspective, that's great. Everyone in the world now knows that we have to fix something. But actually, the time from the disclosure of the vulnerability and having to patch is so short right now that there is so much publicity of the amount of websites being hacked because people aren't keeping up to the speed of the patch management. Right?
[00:13:49] So, kind of to close that point, right, that you're asking. I said, be brilliant at the basics. I said, if you have a rigorous process in your environment, no matter what size you are, if you know that vulnerability is raised, you have a process to immediately patch your devices. And everyone knows your vulnerability management and you know your environment. You're probably in a good place. Right?
[00:14:12] So, look, I think that the round way about it and that question is to say that, you know, the agentic piece and the taking actions does bring a lot more risk. But from an organisation perspective, as long as you're brilliant at the basics, you probably are in good hands. Yeah, so many great points there, especially around identity access management. I've seen so many new start-ups appearing now to try and manage this influx of agents that we're seeing.
[00:14:41] And I appreciate this next question. It is going to be almost a podcast episode on its own. But if we've got somebody listening and they're, before their organisation, grants an agent access to a critical system, we're talking about guardrails, permissions and monitoring controls. What should they be putting in place as standard? What should be non-negotiable right from the beginning?
[00:15:02] I think the first thing you look at it is that if I look at the guardrails, I feel something in my day-to-day is kind of an enterprise AI security model or blueprint. Right? And it essentially, it comes in layers. It's a defence in depth. Right? So, you're talking around different layers around governance. You're talking around aspects around the protection mechanism. And then you're looking at what the AI can do. So, around the decision-making. Right?
[00:15:30] And then finally, it's the control after and the visibility. Firstly, as an organisation, start by putting in a framework and a governance model. And the non-negotiable, firstly, is what use case are you bringing it in? Have you permission to bring that into your organisation? And have you considered the risk around it? Right? Too many organisations are just adopting AI on the fly. If you were in an organisation today and you said that I'm making a big change the way we work,
[00:15:59] that starts from typically the board down, giving investment to the senior management, accepting the risk of the potential new project brought through. And with that risk, you have a financial commitment and a team stood up to do that. However, with AI right now, we completely seem to have missed that. Right? We just think that anyone can be their own developer and they can add permissions.
[00:16:25] And especially potentially in the small medium enterprises where, you know, you don't have the same level of governance. Right? In place. So, to start, if you're utilising agents specifically, because I think agents is where the risks are in this case. You know, we're talking about in that protection domain. What permissions does the system have access to and limit it, firstly? Second one, be very tight in terms of the prompt.
[00:16:53] So, like, if you think about that AI decision making, it happens at a kind of three stages is how I look at it. Right? It's your inputs. Right? So, what systems have you access to? What data has it access to? You have the model itself and the prompt. So, what model has it been trained on and what data? And then the prompt equals the outputs. So, the only thing that you can limit here is the inputs and the decision making element.
[00:17:19] So, you need to bring tighter control around what's allowed and the guardrails in place. And then after, it's monitored, monitor it and have a kill switch. And a kill switch should be that every agent that talks to a system specifically, so the agent should have its own identity. And also, every single connection it's allowed to should have its own specific identity that you can cut off all of them at any point and have that fully controlled.
[00:17:49] So much great advice there. Yeah. And as we've said a few times here, behind the headlines of AI going rogue or agents going rogue is that they're just trying to complete a task. Or over time will slowly drift away from their initial objective. So, how should businesses better test for unexpected behavior when the possible failure modes are so difficult to predict in advance? That's a difficult one, to be honest. You can't predict every failure mode.
[00:18:18] That's the point, as you just said. So, stop trying to essentially predict them all and reduce the blast radius. I think it's back to my last point. Yeah. Assume it'll surprise you. Look at cyber today, right? I think anyone who's been successful in cyber is assuming breach and have the necessary kind of protections and layered defense inside it,
[00:18:41] such as micro-segmentation, really strong identity, so that if their perimeter is breached during lockdown to a specific area, right? And that's controlled. And then monitor and continuously detect what's happening. So, if you're thinking about bringing in an agent, if you take that same philosophy, and one, you verify it within a controlled environment, a sandbox, for example.
[00:19:06] Watch what access you're giving it to it and making sure that its outcome is exactly what you expect. Next, move it towards a shadow mode. As I kind of mentioned to you in terms of my day-to-day job, it's around deterministic and autonomous flows. So, therefore, you have human in the loop. So, test it for a few weeks with human in the loop. And secondly, you shouldn't give a finance, something that processes your invoices and something that pays your invoices the same.
[00:19:36] You can have a human in the loop. Two potential agents doing two jobs, but someone with the guardrails in between. So, make sure that when you're testing the behavior, you know what happens. And then have a runbook and kind of monitor as it grows. See what is happening. See the logic behind it. A lot of people just turn on agents and leave them be, right? That's where, you know, we would have problems to have the line.
[00:20:01] And from an industry perspective, my experience is this whole centralized platform for managing agents. Example, Agent 365 as a Microsoft solution, Times 3B as well in solutions that we work with as well. And they're all new, right? So, you can't assume that every organization knows how to adopt them or has adopted them.
[00:20:27] Because most investments of this nature typically come with a five-year plan. But five years ago, this didn't exist. So, how could this be on a five-year plan? So, I think the basics is start small. Look at that blueprint of keeping isolated, only give it the same permissions and monitor it. And I think you'll be okay. Yeah. And the other big question we've got to bring in here is accountability.
[00:20:54] Of course, in the old days, human tech teams would all point to each other like the Spider-Man meme, saying, it's not my network, it's your application. It's not my application, it's the third party that manages it. So, right now, though, how should accountability be divided between the vendor, the deploying organization, and the people who approved its access when an autonomous agent could cause harm or something unexpected? Where does accountability sit? If we just take AI outside this for a second, right?
[00:21:23] Just think about where accountability and any kind of specific action is. You can outsource the task. You can't outsource the accountability. You know, essentially, if you look at any of the directives across Europe, NISTU, DORA, UAI Act, as an example, the risk is at the management's desk. A regulator won't accept that an agent did it no more than they'll accept the intern did it.
[00:21:51] So, the accountability sits at the business, right? So, that's the first kind of threat, right? Then you look at the agent itself, right? The agent itself. You purchased an agent with clear documented guidelines, right? So, I don't know how the accountability would be in an answer that you are deploying it with your environment. You are responsible there.
[00:22:18] You have purchased a product to do something, and you own the accountability and responsibility. Absolutely. However, I do find when I was looking at the articles over the last couple of days, and I'm not sure if you saw anything around the new AI safety bill that they've, yeah, and tropic pitches to say that the bill should block a model if something like 50 people should be killed, up to 50 people.
[00:22:45] And I can't, like, comprehend that. Like, that is absolute madness, right? The fact that we are saying that they would block a model if a large number of people. I think you killed was the lines that I've read across the news articles, but I think that comes back to accountability.
[00:23:06] Firstly, I think the, in my opinion, the model owners are responsible for one level of guardrails in place, and then the day-to-day use is the organization's management and board. Love it. And finally, for anyone listening, I'd love to give them a few valuable takeaways here. If we have someone listening in maybe an SME, they want to move quickly enough to benefit from agentic AI
[00:23:33] without leaning too much on governance or slowing things down or creating problems further on down the line. Any advice, tips on what they should be doing right from the outset? Yeah, look, I think the first aspect is don't just turn it on and start using it. Start by bringing it up to a board level and management decisions to understand the scope and boundaries that the organization that you work for accept.
[00:24:03] Essentially, that would define the use case that it would be brought in for, and that has an associated risk appetite that has to be signed off from a board level. From that kind of element, that governance forward, you can set a light policy in terms of how it's used. And based on that, you can bring essentially a pattern for use within the environment to say that, you know, okay, once we have this use case, this is how we're going to use it. This is how we're going to implement it.
[00:24:32] And looking at this from a building blocks perspective, like just like you would build an application right today. So if we were to build an application in our environment, we'd have a web app. We pick one of the infrastructure components within our environment like Windows or Linux. We'd apply the hardened image that's approved by my organization, which is in line with the identity and access management policy.
[00:24:56] And every single building block that I described there from that standard build perspective, you'd apply a similar pattern to AI. So define the pattern that can be reusable. And therefore, you're essentially putting guardrails in place for each stage of usage. And that allows you to protect your environment, to control and detect if something goes wrong, and then reuse them patterns over and over again.
[00:25:24] I think that you have to start somewhere. So start by not looking at every single use case. Break it down to a single use case that you find the most value. And from that there, you can essentially build upon it, but at least you have the foundations in place. I think that is a powerful moment to end on. And for anyone listening wanting to find out more about you guys, Echo is one of the fastest growing security first MSPs in Europe.
[00:25:53] So I do urge people to check you out, but where should they go? Where can they find out more information, connect with you or your team, etc.? Yeah, look, our website is geek.co. Reach out to us or you can reach out to me directly. I'm on LinkedIn under Jonathan Traers. Always happy to connect with anyone. And I think from a takeaway perspective, look, AI is moving really fast, right? But so how does every other technology advancement that has came before it? And I think it's by starting small, building a pattern,
[00:26:23] and utilizing kind of small building blocks to be able to introduce that technology into your organization. If you're on the other side of the house and you're worried about how your organization may be compromised from external agents, etc., I think just get really good at the basics. Vulnerability management, patch management, configuration management. There's nothing new here. It's hard work, but that's what's going to keep you safe. Fantastic advice.
[00:26:51] I will add links to absolutely everything. I urge people listening to check you out and continue this conversation we started. And I would also love to hear from you. TechTalksNetwork.com. Give me a shout. Let me know your challenges, the good, the bad, and the ugly out there with all things AI and AI agents. But more than anything, Johnny, just thank you for joining me today, bringing all this to life in a language everyone can understand. Really appreciate you, Tom. Thanks very much, Neil. So much gold in Jonathan's answers today.
[00:27:20] And one of the big things I'm taking away is to look closely at what an agent is allowed to do. A helpful instruction cannot compensate for access that reaches further than that work or request requires. So we're talking about separate identities. narrower permissions and a way to withdraw access and make that responsibility easier to manage. And I think Jonathan also made a case for beginning with one defined use case
[00:27:48] and building a repeatable pattern around it. Test in a controlled environment, keep a person involved while learning how the system behaves and then monitor everything that happens after deployment. What I'm trying to say is the work continues when the demo ends. So a big thank you to him for joining me today. I will add his LinkedIn profile, the website and a few other things over on the blog post associated with this at techtalksnetwork.com. But let me know your experiences.
[00:28:18] And also have a think. If an AI agent surprised your team tomorrow, would you know which system it could reach and who could stop its access and its tracks right now? If you can't answer that right now, then maybe you need to be having a chat with someone in your tech teams. But let me know, techtalksnetwork.com. I'll be back again tomorrow with another guest. Thanks for listening. Bye for now.