What happens when security analysts encounter so many false positives that dismissing alerts becomes an automatic response?
In this episode of The Business of Cybersecurity, I’m joined by Tim MalcomVetter, General Manager of Coalition Security and co-founder of WireSpeed. Tim brings 26 years of practitioner experience, including building and running Walmart’s red team and leading a managed security operation serving hundreds of enterprise clients.

Tim believes the familiar phrase “alert fatigue” fails to describe the full operational risk. When analysts repeatedly investigate events that lead nowhere, their judgment can gradually recalibrate. An unusual event starts to resemble another routine item in the queue. That normalcy bias is especially dangerous when analysts have limited customer context, strict response targets and thousands of additional alerts waiting behind the current case.
We discuss how AI-amplified attacks change the timing. Tim explains that defenders may begin several minutes behind because endpoint events must be collected, processed and passed into security systems before a response can begin. Attackers can combine AI-assisted preparation with deterministic tools that execute in milliseconds once they reach an environment. Even if both sides have access to capable models, the defender may already be running a delayed race.
There is another complication. AI analysis consumes tokens, and security teams rarely have an unlimited budget. Tim expects attackers to use lower-severity techniques that may sit below the threshold organizations send for expensive AI investigation. At the same time, security vendors have a commercial incentive to report broad telemetry because being accused of missing an incident carries serious consequences. The result can be greater visibility, but it can also place additional noise in front of already stretched teams.
Tim offers a practical model for deciding where different forms of intelligence belong. Deterministic code should handle the widest possible set of known and repeatable decisions. AI can interpret ambiguous or semi-structured material, such as suspicious process trees and living-off-the-land activity. Human analysts then provide judgment, supervision, and investigation where neither layer can produce sufficient confidence.
We also examine WireSpeed’s approach to learning from errors. Tim describes replacing the traditional alert funnel with a straight pipe that routes events toward a verdict, while people statistically sample the results. When the system makes a mistake, the team investigates the defect, corrects it and converts the sanitized case into a unit test. That approach aims to stop the same failure from being repeated across analysts or software releases.
AI can help security teams process context, but probabilistic output also creates testing, consistency and supply-chain questions when models change. How should your organization divide responsibility between code, AI and people, and has your SOC reduced false positives or simply learned to live with them? Listen to the conversation and share your thoughts with me.
Useful Links
Connect with Tim MalcomVetter
Learn more about Coalition Security

[00:00:00] The leading issue of agentic AI in businesses right now is ensuring agents act with compliance guidelines. And Denodo applies guardrails across your entire data estate. By aligning your company's data infrastructure under one system, these guardrails perform consistently across your platform. So start scaling your business and start with Denodo. Simply visit denodo.com to learn more.
[00:00:33] What happens when a security operation centre sees so many false alarms that a genuine threat begins to look ordinary? Well, today on The Business of Cybersecurity, I'm joined by Tim Malcolm-Better, Head of Security Services at Coalition and co-founder of Wirespeed. And Tim, he spent 26 years in cybersecurity, including building and running Walmart's red team
[00:01:01] and operating a managed security service supporting hundreds of enterprise clients. But he will argue today that alert fatigue is understating the problem. Repeated false positives can almost condition analysts to dismiss abnormal behaviour as just another routine ticket. While automated attackers could already be moving through systems before an alert even reaches the queue.
[00:01:29] So, yeah, we're going to discuss all that normalcy bias, vendor incentives, token costs, and why effective detection and response could depend on deterministic workflows, AI, and good old-fashioned human judgment, all working at different layers. And we will have a bit of fun along the way. It's a really interesting episode, this one. Tim's a great guest and refreshing to hear him speak so candidly. So let me introduce you to him now.
[00:02:01] So thank you for joining me on the show today. Can you tell everyone listening a little about who you are and what you do? Neil, thanks for having me. My name is Tim Malcolm-Better. I've been doing cybersecurity now for 26 years, mostly very technical roles. Lots of enterprises. Built and ran the red team at Walmart. Largest company in the world for five years. So his job is run a team of hackers to break in basically 24-7. And ran a large managed security services company after that.
[00:02:27] Had a managed SOC with 300-plus enterprise clients on it. Pretty good experience there. And I left there saying, we can go faster and we can do better. And there's another way to go about this. And I left and went to a company called NetSpy, which is one of the largest pen test firms in the U.S. or in the world now. And met my co-founder, Jake Reynolds, there. We built WireSpeed, which is our automated detection and response platform.
[00:02:52] 16 months out of stealth, we get acquired by Coalition, who is the largest cyber insurance provider in the world. And we've just been taking off ever since. WireSpeed just turned two this summer. WireSpeed is a large number. And the last I checked yesterday, we have over 2,500 tenants on it already. So it's going pretty crazy. Yeah, exciting times. And there's a lot I want to be talking with you about today, especially because of your vast experience there. I'll look at some of your insights.
[00:03:19] So when we talk about cybersecurity, one phrase that props up a lot is alert fatigue. And one of the reasons I bring that up is I was reading before you joined me today that you argue that the deeper cost of false positives is normalcy bias rather than just alert fatigue alone. So just to bring that to life, what is it that happens to an analyst judgment after months of discovering that almost every alarm means absolutely nothing? Well, first of all, thank you for reading stuff that we write because sometimes it doesn't get read, you know?
[00:03:48] So that is a very interesting take. I don't like the cliche answers. I've spent too much time as practitioners. I've hated most vendors most of my career, and I hate the most of the marketing. So the alert fatigue phrase is especially poignant for me. I just hate it. If you think about what we do to people, we say, hey, great idea to get a job in cybersecurity. Let me throw you straight into the fray. You should start working in the SOC. And oh, and by the way, probably you should start working in a managed SOC, which means you're not actually protecting a company that you work for.
[00:04:15] You're protecting another company, which means you're going to have a hard time knowing everything that's going on and getting the full context. Oh, and by the way, we're going to deluge you with these alerts. We didn't tell you this exactly, but you're going to have air traffic controller level stress, right? You're landing planes, except for the planes are breaches. And most of them are going to like, you know, planes don't crash very often. So same kind of thing, right? Breaches don't happen relative to the amount of alerts you're going to see. They really don't happen all that often.
[00:04:41] Oh, and by the way, while you're doing this, you're walking this knife's edge because you don't have this context, right? So as you're walking this knife's edge, if you lean too far and you tell a customer about something, you risk making them very angry at you for annoying, for taking their time up with something they think is a nothing burger, right? And on the other side, if you don't tell them about it, you risk, well, that's the breach. And you're now the worst SOC analyst in the world. And they have impact. And your company is definitely getting fired, right? All these bad things.
[00:05:09] And what happens over time is like, that's a high stress situation. And then what tends to happen more often than not is they tend to have more people mad at them for telling them about stuff that's not true the other way. So they tend to get like rapid fire on the false positive button when they're just clicking through this. And if you show this to them alert after alert, and you think about in a human speed SOC, you've got 30 seconds to two minutes maybe to get to it before I need to move on.
[00:05:35] And most of these tickets that come into your queue, right, have a, like you have an SLA attached to it. And they maybe have 15 to 20 minutes left on it, right? So you have 15 to 20 minutes to read this thing, figure out what's going on. And the second you close it, don't worry, there's thousands more. And that's your job until your shift ends. And so when you sit here and you do that over and over and over again, you stop and think about what we're doing to that brain. We're conditioning that mind to think most of the time I need to not tell somebody about something. Every once in a while, something will pop up. It'll be bad. But most of the time it's bad.
[00:06:04] And that's just not a position that most people want to put their brains in. And so it does create, you know, obviously this normalcy bias and it does create the opportunity to miss things. And AI is not going to help it. But I'll let you ask your next question before we jump into that topic. No, I mean, just listening to that, I can almost feel people nodding in agreement around the world, especially cybersecurity analysts going, yes, that's my guy right there. Yeah. Because they live and breathe this stuff.
[00:06:30] And of course, now to throw into that mix, attackers are using automation and AI to move through systems faster. In fact, at machine speed. How does this speed change the consequences of maybe a delayed or dismissed alert? Because it takes it to the next level, right? Well, there's two things. There's the obvious thing, which is if the threat actor gets in fast and there's a human speed queue at the back end of it, and it takes 30 minutes to an hour, and they can get to their objective in 15 minutes.
[00:06:59] Then obviously, there's material impact right away. And there's just nothing about it. Like, that's the obvious thing here. But there's another thing here that I don't think very many people are talking about. And that's what most defenders that are thinking, oh, bad guys are using AI? I'm going to use AI. And they think, yeah, this is great. I'm just going to have the same tools. And they don't realize what you're doing is, number one, you've got a foot race of equals, right? We're both running similar tech stacks now. The difference is a threat actor is going to have a two to five to 10 minute head start. And the reason for that is real simple.
[00:07:28] It's just ingestion delay. If I have a malware event on an endpoint, doesn't matter if it's Microsoft, CrowdStrike, Send One, pick your poison, your favorite vendor, they're going to have an ingestion delay between the ad event happening on that endpoint, that agent on that endpoint seeing it and saying, this is a thing I need to send off. Funneling that upstream to that SaaS instance where that provider actually collects all the data, then it needs to get published and go through their pipeline and come back down to you to your SOC. And then whatever speed you've got to put eyes on or do whatever, you add all that up.
[00:07:58] Even if I just take all the humans out of it, it's probably five minutes on average delay. So now if I've got a threat actor that comes in with state of the art models, and a lot of them, they're running obliterated models. So they don't have the guardrails where you ask Claude to do something and Claude goes, the United States government told me I can't tell you how to do this. Right? So, you know, they're just getting the answer straight away for one. So you've got that piece of it.
[00:08:20] So you've got Usain Bolt, who's got a five minute head start, and you've got allegedly Usain Bolt, who sometimes gets handicapped by various reasons, running five minutes behind that. Right? And if they're both running the same speed, the obvious thing is the bad guy's going to win. Right? Like, yeah, that's the thing that's coming up. First of all. Right? So we've got that. The second thing to keep in mind is that threat actors aren't always going to run completely agentically for everything they do.
[00:08:45] And so we're going to nerd out for, we're already nerdy already in three minutes of this thing, but let's get even more nerdy for a second. So in computer science terms. So when you're running something in AI, you're running non-deterministically. If I'm writing, let's say a Python script and it runs, that's running deterministically. What everyone needs to understand is that when bad guys are using AI, they're using it to go do things like, I'm going to go build an exploit for something that doesn't, it's not known out there. That's what everyone's afraid of. Right?
[00:09:11] Mythos, the second that came out, everyone's like, ah, we're going to find all these vulnerabilities and run and write exploits for them. Well, that exploit, when it runs, is deterministic. It was non-deterministic, meaning slow to build, faster than humans, but slow to build. But the second it hits your environment, the second it matters, the second you make contact with an enemy, it's running deterministically. And then once that runs, what's that going to do? It's going to give them an access, give them a shell. When they have that shell, they're going to then immediately go, where in the world am I? I found this space. Where am I?
[00:09:40] I need to figure out, I need to orient myself. I need to do some discovery. And if you stop and think that a threat actor is going to be dumb enough to go, hey, Claude or whatever model I'm using, can you just go make a plan for what to do next? Like you're mistaken. Smart, like threat actors that are going to do all that other work are going to be smart enough to go, nope, I've got a pre-planned set of things. I'm going to go execute deterministically, meaning they're going to run in milliseconds. So you've got that. All that happens somewhere along that trigger.
[00:10:08] That's where that first EDR alert happens, if you're lucky, right? I've got an endpoint alert that says, and then that's now off on its two to five to 10 minute delay, whatever that delay is. But I've already started my discovery and I've already started a bunch of stuff deterministically. Then as an attacker, only then am I getting it fed back to my AI. Then am I running non-determinously? And why does that matter? Only then am I slowing down, right? I'm millisecond speed. Now I'm seconds to minutes speed. And if you come back in and we've got threat actors going, oh, I'm just going to take our defenders going, I'm just going to take this stuff.
[00:10:37] I'm going to throw it to my AI sock tool or maybe I've got some prompts and scripts that are running against Claude or whatever, GPT. And I'm going to go do what I'm going to do. And it's going to go make a plan for how to investigate this. Well, you're five minutes behind and you're inventing a plan. You don't know how to work a case. Like, this doesn't make any sense to me. But yet, this is where the market's going. So you've got that whole thing. And then a third level of this, Neil, which is why this space is so interesting to me, is we have token consumption.
[00:11:05] And the companies, the defenders that are actually starting to adopt this are realizing, well, before AI, let's just back up to 2023. For a second, right? 2023, the state of the art was I have to round robin all of my alerts and put them into a priority queue. So I get the most important things to a human because I have a compute problem, but it's human brains, right? Human compute brain. I've got to filter this stuff down and I have to have the right human get to the right thing at the right time. And like, that's what managed socks are all about.
[00:11:33] And like having elasticity, just like you would have elasticity with like a computer or whatever. So you got to, we got to go do all that. Well, now in this world, we're doing the same filtering, but instead of filtering what goes to human, we're filtering what goes to the AI because I don't want to burn tokens on everything.
[00:11:48] And I think what's going to happen is the smartest threat actors are going to use AI to go build initial access tools that run deterministically once they make contact that also then chain together a series of TTPs or techniques that all run in the low to medium range by most EDRs. Because what's going to happen is you're going to have enterprise defenders going, but don't worry. I bought, I attempted to drop 10 names right now of all the different top AI socks. I run these and asterisk.
[00:12:17] I can, I have to pay per investigation. It's a consumption model, or I have a capped quantity of investigations I can run. So I'm only throwing my criticals and highs to it. And so what's going to happen is they're going to land. They're going to move really freaking fast. And they're going to go using lower level techniques that take longer to get there because it doesn't matter. You're not going to look at them. That's where I think the future is. And that future is like weeks to months.
[00:12:40] And right in the crossfire of everything we're talking about here, there will be security vendors that will paint themselves as the good guys and increase alert severity because, hey, they don't want to be accused of missing an insurance. How does that incentive just create even more noise that attackers can almost use as cover? Yeah. Yeah. It's a really interesting problem. So if you are, it doesn't matter the vendor, I'm not going to even drop the names. We all know the names.
[00:13:06] If you're one of these vendors and one of your clients has a breach, the last thing you want to have happen is that client to say, Mr. Vendor, you missed this thing. You didn't see this. You didn't detect on this at all. I get nothing out of you. Right. Yeah. And so they're first of all, the top incentive is to instrument everything and tell you about everything. Right. They try to tell you critical, high, medium, low, whatever, but their real incentive is just to make sure you've got signal on it. Most of these same vendors have prevention capabilities.
[00:13:34] This is really important to understand this nuance of these incentives, right? Why do I bring that up? I bring that up because if I'm vendor X and I'm the state of the art best endpoint identity cloud, doesn't matter what I am. If I see something that is so bad, why didn't I just prevent it? Right. If I see something that's so bad, I'm going to prevent it. But if I'm like, I'm not sure, I'm going to just tell you about it. I'm going to tell you about everything about it, which means they're going to over index everything. They're going to over stimulate you and send all these things to you.
[00:14:04] And so that just compounds the problem. So you combine the fact that in the last, call it five to 10 years, EDR adoption is really, really high. Cloud adoption is really high. The more things that are containerized means the more things that are instrumented to the moon. So that means more telemetry, not less. Ever since the adoption of cattle, not pets for infrastructure, if you remember that terminology, we have we started down this path where we're on this like upward trajectory of the amount of telemetry that's going through. So it's not going to get less anytime soon.
[00:14:33] So basically, the net of it is human capacity can't scale. And it's going to have to be machine versus machine, but it has to be done in a different way. It can't be the same status quo that everyone else is recommending today. And elsewhere, AI first security agents, they promise contextual analysis at scale. That's the sales pitch there. But again, I was reading that you've questioned whether probabilistic models should make binary containment decisions.
[00:14:59] So where is AI useful here and where is maybe determinism the safer choice? The answer really is, I mean, I say this all the time to people. I say it to my team. I say to my customers. I say to everybody. There's three levels of things you can do. The most expensive thing you can do in security is to have humans do the work. The second most expensive thing you can do is have AI do it. The least expensive thing you can have done is deterministic. And so the answer is I should my foundation.
[00:15:25] If I was building a pyramid of it, it should be like very wide, very deep deterministic for almost everything you possibly can. And then AI should come over the top. And then on the top of that is humans. And so why do I say that? Well, because if you're looking at things, the first problem with doing deterministically, that means you have to understand the domain really, really well. Like you have to understand the entire body of things that can happen. You have to write code for all the edge cases. So it's a lot of work, right?
[00:15:49] And one of the things I'm picking up a vibe on, and no pun intended with vibe and vibe coding, but one of the vibes I'm picking up right now is a lot of people are lazy. Right? And a lot of vendors are lazy. And AI seems easy. Right? One of the problems when I talk to AI sock vendors, they say, well, one of my top competitors is the do-it-yourselfer inside the enterprise that thinks, oh, I've got a cloud license. I can just go build what you built. And like, because I think I can vibe code a user interface, a web interface, right?
[00:16:17] And I can vibe code whatever, and I can tweak with prompts. And there's this idea that I can tinker. And this has been actually a problem with enterprise security for some time. We've had, and I could trace all the way back 20 years ago when we first started shipping the first security appliances to enterprises. They came with no content. So we expected the network engineer who was not a security specialist to start creating detection rules and content for this. And ever since then, we've been on this do-it-yourselfer thing.
[00:16:43] And it doesn't make any sense because at the end of the day, I've literally talked to senior and principal level engineers inside of enterprises who think that they by themselves can go with their Splunk rebuild CrowdStrike. Right? And that was the attitude five years ago. Now with AI, it's gotten worse. It's like this attitude of, oh, I could just go build all this stuff out, and I can just go chain it all together. And where the context stuff is great, it comes at a cost with tokens, which means you can't throw everything at it. And it's not infinite. Like everyone has a budget.
[00:17:13] And it doesn't matter how much the token costs come down. They will never, ever scale. Like just physics of it will never scale as well as deterministic. So you need to have that first. And that AI overlayer is good for things like, I'll tell you places where we use it. Live off the land execution is a great place for it. Why? Well, because it's usually a bunch of sketchy commands that are all kind of chained together. So you've got process trees. So it's a bunch of semi-structured text, but it's not super structured. And that turns out large language models are really, really good at that.
[00:17:42] So that's a good place to go take that and break it up and basically help me figure out how to classify it better. And then we use determinism on the back end to go figure out, okay, based on this, I now know exactly what I want to go do. It's not perfect, but nothing is. At the end of the day, security has to be layers and you have to have a layer down below that's ready to fix the one that failed above it. But I tell you, I mean, like, I will ramble on this topic for hours, but the gist of it is, Neil, that AI is not going to save us, right?
[00:18:10] It's not going to go make things automatically better. And I think we're starting to see practitioners realize this, right? I would tell you a year ago, well, the attitudes around AI have changed so fast last year. Two years ago, I think there was a small cohort that pretty much was like, this is going to be the thing that changes everything. I think a year ago, everyone felt like if you didn't say that, you were going to get banned from your career, right? Like you needed to be on that bandwagon.
[00:18:37] About six to nine months ago, that started to turn and I started to have people whisper to me like, I agree with you, but I can't tell you. But I don't want to say it on LinkedIn, right? Like that sort of thing. And now I think we're starting to see people who have gone through the exercises. They've had boards that, you know, they've said, what's our AI story? And you need to be spending more on AI across the entire company. And here's some budget that we never previously had and go spend some of it on cyber. And one of the first obvious places is to go apply it in the SOC. And then they look at it and go, yeah, this is maybe helping a bit and it's exciting. And they'll do press releases.
[00:19:07] But then the practitioners that are in there going, yeah, but I still have to double check it. Right. And it's like having a junior employee that has like extreme capacity to do work, but also it doesn't have the same judgment that I do. Wow. I mean, it's what, 4,000 miles between us. And I think we've both heard CEOs go, what's our AI story? Where are we in the AI narrative? And it's just completely the wrong thing to say and the wrong approach. It is the wrong thing. Yeah. Yeah.
[00:19:32] And of course, at Wirespeed combines deterministic workflows, AI and human expertise. And just to bring this to life, could you just describe how these three elements work together during, let's say, a real detection and response? I mean, it's exactly how we kind of described it. So it's kind of like if you're building a pyramid and the base of its determinism, the vast majority of things that we're going to go consume, we're going to know how to investigate. And so it's this really simple philosophical difference of we've worked in a SOC.
[00:20:02] We've built SOCs. We understand security. I've created incidents as a red teamer. I know how instant response people think. We've forced them to be put in a weird spot. So we understand the space so well that for us, I didn't need to go turn and say, like, if we're going to build something like this, I don't need to go ask AI, please build me a plan for how to investigate this. We have the plan for how to investigate this. So I don't need to waste that first step and all those tokens around it and the latency and the ingestion costs that come with that.
[00:20:29] So the majority of everything we can do can actually go into it just goes into a taxonomy. We know what kind of event it is. What's what category is it? That category immediately translates into what we do with it. And if you think about this, Neil, this is no different than in theory how SOC teams have been working for 20 years. You can say, Neil, welcome to the SOC. Here's my SOP documents that are probably pretty shoddy, but they're here. If you look on this, when you get endpoint detections, I want you to go do these steps, figure out basically classify it.
[00:20:59] What kind of endpoint event is it? And if it's a, you know, I'm kicking on live off land, but if it's live off land, we're going to come over here. I want you to check for these things. If it's a persistence thing, I want you to check for these things. And we build these SOPs for how we want our human analysts to do it. If you stop and think about it, that's a prompt for human. Sometimes they're pretty well thought out. Sometimes they're not. Sometimes they're missing, like almost every time I've ever seen this, they're missing edge cases.
[00:21:24] And they may be incomplete in as much as an enterprise only sees, you know, events in their space. So if they don't have a breach, which most enterprises don't have a breach every day, you don't know the right way to do that end to end. You're basing it on basically campfire stories from somebody else who's gone through a breach or from somewhere else you've worked or best practices that get kind of get passed on to like almost like oral tradition. And what we're doing is we're basically up leveling us from the campfire oral tradition up to into something that's written in code. And that's so that's effectively what it is.
[00:21:53] We're saying, hey, guess what? That SOP document for day one, Neil, that's actually an algorithm. If you stop and think about it, and since it's an algorithm, that means I can write code to implement it. And that's what we've done. And so we do that for the vast majority of it, for the things where we say, OK, I can't get the level of precision that I want on these. I will go and use AI for certain pieces of it. And then we wrap. We have the whole agentic AI thing. If you want a cloud or chat TPD like interface to your data to go ask questions, you're more than welcome to. We have all those features. We don't even demo them hardly in sales calls.
[00:22:24] And the reason why we don't is because we are philosophy is if one of my customers gets to that point where they're using that, we failed. Right. Upstream, we should have done something better. If you're getting to that point. So our primary use case for that is two things. One, to answer deeper questions, just to alleviate again so that customers can get answers quicker than getting into a human because that helps with speed and cost and everything else that we talked about. But also we use it because we instrument all the questions. And then we say, OK, what are customers asking? They're asking for these things. Guess what that means?
[00:22:51] That means we're not delivering it to them in a way that's clear and concise. Let's fix the platform. Let's either do I'm I missing a step? Am I missing a data point? Am I just not showing it to them? So we use it for customer insights more than we use it for anything else. And that to me, I think, is the right approach to go there. And then, of course, the human is extremely critical. But we've completely reinvented the model. So we've been rambling here for a bit. But SOC, for the last 20-something years, has been a funnel, right?
[00:23:20] We throw endpoint, identity, cloud, network, all these different signals into it. We use tools like SIM and SOAR and other, and I like to call them START-ER, EDR, cloud detection response, ITDR, whatever. We use those things to curate fine signals that come to the bottom of that funnel. And then, historically, we've taken a human at the bottom of that funnel to take the item off of it and say, okay, does this go in the yes bucket or to go in the no bucket? That's basically what they're doing, right?
[00:23:48] Well, again, landing the planes, knife sedge, getting customers mad at them. Like the whole thing from the beginning of this conversation, they're doing that off the bottom of this funnel. What we did is we turned wire speed into a straight pipe. So events come in, all that technology comes in, it immediately routes to the yes or the no buckets. And then what my humans do is we reach into the bucket, we randomly statistically sample that bucket and pull samples out and say, should this have gone here, yes or no? And if the answer is no, we stop and we figure out why. If there's a, do we not extract something?
[00:24:17] Do we miss, something is not parsed? Do we not have an edge case? Do we need another step in our investigation? Like what was missed about this? We take that, put it on the microscope. We go fix that particular issue. That detection that went to the wrong place is actually sanitized of all customer data. And it becomes a unit test. So every single time we deploy to production, which is on a random, like today's a Wednesday, on a random Wednesday, we'll probably deploy four or five times.
[00:24:43] And so when we do that, we're running through every single mistake that wire speed has ever made and making sure it never happens again before the next release of code happens. So compare that to a human speed sock where you've got Alice. We hire Alice. She comes in. She's great. And then she makes a mistake. And we go, Alice, this was a mistake. Let me explain why. She goes, I get it. Bob, hire Bob in here. He makes that same exact mistake a week later. You sit him down, coach him. Da, da, da, da. You get to Charlie. Charlie makes that same mistake again.
[00:25:12] Charlie makes the same mistake again and again. And you're like, dang it, Charlie, you're fired. Right. In our world, we only have the Alice the one time. And that's why our precision numbers are what they are. And it's also why we literally put our critical defect rate right on our website. Because A, we actually know what it is. If you stop and think about a traditional sock, they don't know what their critical defect rate is. Like how often are you absolutely wrong and miss something? We absolutely know because we statistically sample. And when we find a mistake, we look for all adjacents that would have been the same mistake. And so I can tell you when that happens.
[00:25:40] And also that just makes the platform more and more precise because we're honing that edge. So every time it happens, we fix it and that mistake doesn't compound. You can't do that with probabilistic AI. That's the real challenge. Because the second you're using an AI model to go do all this stuff, you have to go basically take every permutation of random detection you've ever seen. And you have to run it through a testing harness and say, okay, if I change my prompt, rerun everything through here, burn a bunch of tokens to run a simulation to find out, do they all map where you wanted to go?
[00:26:10] Yes or no. And that's what all these companies are largely doing is they're building these crazy testing harnesses to run over and over again and make tweaks. And I make a tweak for this. I want to handle this edge case, but dang it, that has a regression that pops this thing up. And it's just a hairball. And so they have to do it to the point where they have an acceptance level. It's not ever going to be 100%. It's going to be 99s, something in there. And that's what they have to do to ship. And then the model can change on our, like we cyber people like to talk about supply chain and supply chain attacks all the time.
[00:26:40] But the AI model that's running inside your sock is a supply chain vector because that model changes, right? Like Anthropoc can make a tweak to it under the hood without you knowing. And if that happens, then the outcome changes. So it's a big deal. Wow. And I think that is a powerful and thought-provoking moment to finish on. But for anyone listening, wanting to dig a little bit deeper, find out more about yourself, coalition security, wire speed, or anything that we talked about. Because I think your honesty really brings this to life today.
[00:27:08] And hopefully we can get a few people involved in this conversation, continue it together. But where should they go? Where's the first port of call? Well, of course, you can go check out wirespeed.co and coalitioninc.com. You can check those, our websites out. You can find me on LinkedIn. I promise you, I'm the only Malcolm that you'll ever find out there. It's a unique last name. So I used to hang out on X a little bit. I'm not really there all that often, but those are all the typical places. But yeah, and I've got a great team. Ping us, hit us up. We give out free evals for customers.
[00:27:36] Let them go spin up a tenant of wire speed and go run. Go see it for yourself. Because the thing I always tell people is like we show great numbers and their numbers as a practitioner, I wouldn't believe. I just, I wouldn't believe them. They seem impossible. But we actually run them and we show the customers that we can do those numbers, that kind of performance, that kind of speed to verdicts inside their environment. So Neil, it's very much appreciated to come on and let me ramble on about some topics that are very nerdy and that I'm very passionate about. Thanks. Yeah.
[00:28:03] Well, one of the things that came out today was your passion and excitement for the industry. And obviously your time in the industry as well. You've lived and breathed a lot of this stuff and your tolerance to no BS in there from some of the marketing claims out there is refreshing to hear. So please, everybody listening, check out the links in the show notes. Spin one up for free. There's your challenge there. Let me know. What did you find? What did you like? I'd love to hear from you. But more than anything, Tim, thank you for joining me today. Really appreciate it. Yeah, thanks for having me. Thanks, Neil.
[00:28:32] I think Tim left us today with useful hierarchy for security operations. Put deterministic workflows at the broad base for decisions that can be defined in code. Use AI where messy context needs interpretation and keep people at the top for judgment, review and the cases that resist easy clarification. And that model also changes how a SOC learns.
[00:28:59] Instead of asking every analyst to remember every previous mistake, Tim describes turning errors into tests so the same defect can be identified before the next software release. It's a different way to think about speed, consistency and accountability, especially when attackers already have a head start. So a big thank you to Tim for joining me today.
[00:29:23] But over to you, is your security team reducing false positives or merely just becoming accustomed to them? Love to hear your thoughts as always. TechTalksNetwork.com. That's where you'll find over 4,000 episodes across eight podcasts that I host over there. And you can send me an audio message or meet me at a tech event. But that is it for now. I'll be back again real soon. Bye for now.

