How LaunchDarkly Is Helping Enterprises Control Shadow AI in DevOps

[00:00:00] A big thank you to Denodo for helping me make more than 60 monthly interviews possible across the Tech Talks network. And as businesses move from Gen AI to Agentic AI, trusted data becomes everything. Everything from Gen AI to Agentic AI, Denodo is helping organizations build intelligent, secure and scalable AI solutions with data access, governance and explainable results.

[00:00:28] So build AI that you can trust and do it with Denodo. And you can learn more by simply visiting denodo.com. What happens when AI starts helping developers move faster than the business can keep up with? But most importantly, who carries the can when the code causes a problem in production? Well, my guest today is the CTO at LaunchDarkly.

[00:00:57] And he's going to join me in a smart, lively and very timely conversation. And we'll have a little bit of fun along the way today because we're going to talk about shadow AI in DevOps, the growing use of AI written code and the uncomfortable questions that many teams are just starting to ask. And by that, I mean everything from public LLM and unsanctioned coding tools to compliance headaches, runtime controls, kill switches.

[00:01:27] Yeah, we're going to talk about what happens when convenience meets accountability. And Cameron brings with him a real world experience, sharp insights and a few great stories along the way, including why you can't fire an AI model when things go wrong. So if your teams are moving fast with AI or thinking about it, this one will give you a lot to think and talk about. But enough from me. Let me introduce you to my guest now.

[00:01:56] So thank you for joining me on the podcast today. Can you tell everyone listening a little about who you are and what you do? Sure. I'm Cameron Netazati. I'm the chief technology officer at LaunchDarkly, which is a company dedicated to shipping software quickly and safely. We've been around for quite a while. We think we're essential as the AI development lifecycle has reinvigorated and completely changed the evolution of shipping software.

[00:02:26] I've been in this industry for about 30 years, not the industry I intended on showing up to by any means. I was originally a biochemist and a chemical engineer of all things. So I was off to study medicine or maybe I jokingly say I was off to do drugs as a chemist. But I ended up in software at Microsoft in the 90s. I worked on Windows 2000. I guarantee most people in the world have probably used code I've written.

[00:02:53] Tours at Amazon, Google, IBM, SAP, Ticketmaster for a little bit, where we did some really interesting e-commerce stuff. So basically all over the place. Spent some time as a CIO of a public company as well. So I like to be able to bring that IT perspective into what has traditionally been a product-focused world because those are our customers a lot of times. And I've got to ask, there's a slight gap in your story there.

[00:03:20] Tell me about that pivot that took you from biochemistry into tech. Because very often, I speak to so many people on here, and there's always that moment of serendipity or a little nudge from the universe in the right direction at the right time. But what was it that made you pivot into tech? You know, Microsoft actually found me. Now, I've been a nerd since I was a little kid. I grew up, believe it or not, with some interactions with some famous AI researchers in the 80s out of MIT.

[00:03:50] And just been fortunate, I think I would say, to have had exposure from a very early age to the Commodore PETs and the Apple IIs of the world. So it was always something in the back of my head. And then I ended up working on some software that did 3D structural modeling of proteins from X-ray crystallography. And some folks at Microsoft heard about me in the 90s and just asked me to come out to Seattle and interview. And I fell in love with a beautiful city, beautiful greenery.

[00:04:18] And, you know, I believe they were people who were going to change the world. Software was going to eat the world. And it was an exciting time to be there. And, you know, we did. Everybody loved some of the products at the time that we shipped, like Windows XP, Windows 2000, you know, truly game-changing pieces. And so my entire career, I've truly tried to work on things where my passion's been for changing the game, whether it's how we deliver software, how we deliver experiences, you know, how we build for massive planetary scale.

[00:04:47] It's super exciting to me. Absolutely love that. And both you and I have been around long enough to have seen the early days of the Internet, the move to cloud, the rise of mobile. And then came BYOD, as everyone wanted to use their own phones and own iPads in the office. Then came Shadow IT. Now we've got AI. And most recently of all, it's Shadow AI now.

[00:05:15] So from your perspective at Launch Darkly, what exactly does Shadow AI look like inside a modern engineering team? And why is it becoming such a pressing governance issue for enterprises? Because many people listening probably would have seen it on their LinkedIn or in their news feed. But tell me a bit more about that. You know, I love the fact that people are excited about this. You know, change is inevitable.

[00:05:42] I think we've learned something in that, you know, this industry as it reinvents itself has said that Agile was, in fact, the very manifestation 30, well, 25 years ago. You know, it came back of embracing change as a true element of the only constant. Shadow AI is just the modern manifestation of that.

[00:06:00] Shadow IT, long created risks around, you know, inefficiency, duplicate spend, security exposures, compliance issues, all of the nasties that both your corporate legal team and your CISO and your CIO all keep, you know, stay up at night. These were the things when I was a CIO that gave me heartburn was I used to joke I was, you know, one security breach away from the unemployment line. And I think a lot of CISOs, CIOs feel that way too every single day.

[00:06:29] And we just see this pattern repeating itself with AI, especially when developers use public LLMs and plugins to move faster in the day-to-day work. I am, you know, 100% for people getting personal productivity up, you know, getting rid of the drudgery, the toil. It doesn't do a developer any good at all to write the same git command 50 times a day. And in fact, most of them are pretty sloppy about the comments they put in for their check-ins as a good example of that. And the AI doesn't care. It'll write a really nice, clear, concise statement.

[00:06:59] Here's the key difference, though. When you look at Shadow AI, AI is shaping code and decisions that most of the time go directly into production. They're not just supporting workflows. And when you're left without an audit trail, it's really difficult to explain how something was built or, in many cases, who's even accountable when something breaks.

[00:07:18] A lot of these tools are moving so fast that there's a sort of missing, I'm not going to say regulatory framework because those exist in many cases or, especially in the EU, are starting to really get codified pretty strongly. But from a position of auditability from these companies, they're ignoring them. They're just moving so fast that they're like, we'll get to that later.

[00:07:42] And so you get things like recording tools that don't garner two-party consent in many places where that's required, for instance, or leak internal confidential information. When I cut and paste something into my personal chat GPT account and it's got corporate financial data, who knows where that's going to show up?

[00:07:59] We've seen incidents over the last couple of years, particularly in the last 12 months, where we see corporate secrets, things that are intellectual property that are hard to protect, just leaked out into the public by simply asking an LLM as another character. And that's really dangerous. You know, as we look at IP law protection, for instance, trade secrets are the least protected and people are just willy-nilly throwing them into production. And that frightens me. That's because that's our secret sauce. That's our competitiveness in this world as a company.

[00:08:29] Yeah, I'm nodding in agreement with you there. And I can almost feel people around the world watching and listening, nodding in agreement as well. And I suspect we've all seen and know developers that are experimenting at the moment with tools like publicly accessible LLMs to speed up coding or decision making. And beyond the techies, every department in every organization are doing the same thing.

[00:08:53] So where does that line sit between helpful experimentation, wanting to get involved, excited about the technology and drifting unwittingly into risky behavior that could expose organizations to security or compliance issues? Neil, that's a great question. I still think as I go back to my head of security here and other places, my HR department, like all the trainings we went through, don't do this.

[00:09:21] Don't, you know, and we all click through them. Let's face it. Like these days, I just asked Claude to write a bot to click through for me. I'm going to be the first to admit it, right? And I'm sure you do too, right? But we haven't gotten the message and the training to sink in like we've done around phishing and we've done around, you know, other sorts of, you know, harassment and all these other sort of bad behaviors in the corporate workforce because it's too new.

[00:09:47] I think personally, AI can meaningfully boost productivity when it's used in a controlled and transparent way. In fact, we did a study at LaunchDarkly. We found that 94% of the teams we surveyed actually report faster development. But 91% of them say they've been more cautious about pushing those AI accelerated changes into production. And that line is really crossed when developers start inputting sensitive code or data into tools that haven't been approved or secured by the organization.

[00:10:16] And it's, to be honest, it's very subtle. So you can sit there and say, I'm going to write a tool that exports data from A and just moves it to data B in a pipeline. You know, we do this from our European instance into our American instance, you know, of our data platform for things that are business critical. And I'm super worried personally about, you know, being GDPR compliant, DORA compliant, all the things that, you know, our European customers expect of us. But you write a tool and it puts a data field in there and you didn't analyze that data field, right?

[00:10:45] Oh, it works on both sides and you didn't do that scrutiny. That's really dangerous. It becomes really risky when these AI generated outputs are used in production without the proper review, testing or understanding of how they were created. You know, more from this survey, like I love the data we gathered here. We've got 81% of the teams that knowingly ship risky code because of deadline pressures, right? It's that developer who goes, yeah, it works, right? And they don't know any better. They're not being malicious, right?

[00:11:14] They're just trying to get their job done like we all want to do every single day. 83% of them say that the releases containing the AI generated code are somewhat or very likely to cause those production issues out there. So, you know, it's a scary time. And if you don't have the visibility or governance around how these tools are being used, then this becomes like an immediate compliance security concern.

[00:11:39] And we get 38% of those teams spending more than a quarter of their time resolving incidents, which means faster code is not always more stable code. As an experienced engineering manager, like I think about how teams break down their work, right? I want healthy organizations, number one, because I've worked in some companies where they're not about healthy organization and I'm not going to name names.

[00:12:03] But, you know, I believe in hiring great people who want to come, want to work with me, want to do a good job and are dedicated. Like I assume positive intent every day. I think as part of that, about 25% of the work of a team is generally what I call keep the lights on work, right? It's bug fixing. In the U.S., you don't get a tax credit for this. You can't, you know, get a write-off because it's not risky. Whereas new development work gives you a tax credit in the U.S.

[00:12:34] That 25% is sort of my bellwether of is the team moving fast enough or moving too fast? Because if that's up, you know, let's say it pushes to 50%, they're throwing stuff into production that's just unsafe and they're creating their own problems. And then if it's too little, they're not moving fast enough at all, which means they're not really creating what we would call tech debt over the years. So even with safeguards in place, like 99% of the teams will tell you they have safeguards.

[00:13:00] 70% of those teams still roll back changes at least weekly, which says there's a huge gap between protection and real world outcomes. And these frequent hot fixes are just more or less becoming the norm now as this code goes ever faster. And then, you know, then we hit other bottlenecks, like human scale bottlenecks, like reviewing the code, you know, analyzing what's going on, debugging. A lot of that's still happening at human scale. So I think what I would take away from that is, you know, a lot of people are saying software's free now.

[00:13:31] Code generation has made software free. And that's not true. Software has made code generation free. But all of the day two behaviors around that and the validations and the verifications, that stuff still exists. It still takes time. And now all you've done is, if I'm going to borrow the work of Eli Goldratt, you know, you've moved the bottleneck in the pipeline. I love the goal. It's one of my favorite books the last 40 years, I think.

[00:13:59] And you made such a good point there as well about the pressure that developers are feeling. I mean, it was, I don't know if you saw the viral video recently of Jensen Wang talking about how he'll be measuring performance by how many tokens the team are burning. And if they're not burning that much, they might appear on a radar. Did you see that? What did you take away from that? You know, I, we had our board meeting, you know, last week. And we were talking about this, you know, with the board. And they're asking me too, what do I look at?

[00:14:28] How am I measuring developer productivity? There are companies out there, and this is not anything I'm measuring internally, to be clear. But there are companies out there that expect to spend two times a developer's base salary in tokens per developer per year. So if I have a, let's just say a $300,000 Silicon Valley, you know, mid-level developer base salary, you know, somewhere in that range. Right. They expect to spend $600,000 a year on tokens for that guy or girl. Wow. Right.

[00:14:57] That's crazy to me. Because are you really getting the productivity out of that at the other end? Yeah. And, you know, what I'm seeing here is you're getting the productivity, but you're getting the rework that goes with it. And that, to me, is, you know, if you go back to lean, it's waste. Right. Yeah. In the process. Yeah. 100% with you. And another stat I read elsewhere recently was that 70% of teams are now using Shadow AI tools to get work done.

[00:15:24] And I mean, that just shows you the scale of what we're talking about here. But when teams are using unsanctioned AI tools without clear oversight, organizations obviously can lose visibility into how decisions are being made or how code is generated. But what kind of risk does that create for engineering teams, especially when there are no audit trails, runtime controls, or clear ownership of the final output? The me of 20 years ago finds that stuff quite scary. Oh, that stuff's missing.

[00:15:53] But tell me more about what you're seeing here. I get to be a little snarky because my mother and her family grew up in New York, so I get that little bit of underlying sarcasm in how I look at things. You can't fire Neil an AI model. You just can't. I can fire a human. I mean, that's the snarky way of saying it. But what it really boils down to is it creates an accountability gap. I actually, I mean, to be blunt, I don't want to fire a human.

[00:16:20] I mean, I value the hard work my folks put in and everywhere I've been, it's been that way. But you can't have an accountability gap in a healthy organization. And it didn't matter. This was, I mean, before we had AI in the organization, there were a lot of organizations that had no accountability, right? And this is where the business starts to falter. But with this, you know, it gets even more critical. Teams can't trace who made a decision. They can't trace how output was generated.

[00:16:46] You know, in the context of the EUAI Act, for instance, you have to have that, right? That's a huge risk to global revenues if you don't have that kind of traceability and that decisioning. And it's really hard to justify decisions around compliance, instant response, and to be honest, the most important thing, which is customer trust in a business. And then you get a security risk. You don't know where these tools are coming from.

[00:17:13] We're seeing poisoning events in some of these LLMs and some of these repos, you know, open source repos out there that are coming about from this. If there's vulnerabilities that are introduced because this version got pulled down by an LLM and nobody checked it, it just works, right? It goes and selects the best tool it thinks it has. There's no clear understanding of those origins. That's a huge risk to my business and to my customer's business.

[00:17:35] And, you know, I say honestly to my folks, like, I don't care if we leak our data as much as I don't ever want to leak my customer's data because I value that trust and that reputation more than I even value, you know, my own business because that's what we stand for. And without runtime controls, you know, you get very flawed or very risky AI-generated code that goes to production unchecked at a prodigious rate that we've just we've never seen before.

[00:18:03] And four and a half thousand miles away from you on the other side of the pond here, the EU AI Act is beginning to move and especially go from discussion to implementation. So how does shadow AI complicate compliance for organizations that are operating here in the UK where I am and indeed across Europe? How do you see this evolving? I think it's really interesting. You guys got you guys I mean, I say you guys, but we sell in Europe.

[00:18:33] So we're there, too. Let's be let's be blunt. Yeah, get a couple of big deadlines to come up. Right. The next one, I think, is early August. Right. The second this year where there's a majority of those rules coming into force, including rules for super high risk AI systems under Annex three, which are the transparency obligations and then all the full enforcement. And you've got to get these regulatory sandboxes. There's a lot of dates driving that roadmap.

[00:19:00] But, you know, it boils down to that transparency, traceability and accountability. And all of that breaks down with shadow AI. If you don't get visibility to those decisions are being made, how are you going to have those high risk compliance? How do you know you're in compliance with those high risk actions? I think when most organizations come in and they start evaluating the risk classification across their organization, they're shocked. Actually, I'm American. I can I can make the joke of shock and awe.

[00:19:29] They have shock and awe at how many things fall in that high risk category. And then that fines those fines that noncompliance risk just goes through the roof with legal exposure and pretty hefty. I think, you know, everybody should be aware of what falls in that high risk commercially relevant bucket, which is anybody using AI to screen, rank, match candidates.

[00:19:53] You know, AI for credit scoring, for education, for critical infrastructure, law enforcement, health care. This is that full compliance regime. You know, and so I'm looking to build systems around that help folks manage risk. And, you know, I can't manage it up front. I'm not a frontier model builder. Thank you know, in some ways. Thank goodness. I'm not a frontier model builder because then I then I would have been under this week, you know, months, years ago, two years ago, I think. But my goal is when you discover something. Right. Let's help correct it.

[00:20:23] Let's shut it off right away. Right. Let's get the remediation in place. So one of the things I actually, you know, like and, you know, I've been around doing I lived in Europe for a while in the in the Nordics in Stockholm. So I'm familiar with sort of the climate. One of the things I really like about this law is is they're really about shepherding you toward compliance. Right. If you sign up, you sign the you sign the compliance pledge. Right. You do all of that. Then they're more focused on remediation and getting this right.

[00:20:52] So the use of AI is healthy versus, you know, punitive. Yeah. And I think, you know, while the fines are large and scary, I think the push is generally in the right direction for how do we use this safely so it benefits everybody that we're doing business with. And so I admire that. You know, in the U.S., we're still dealing with a patchwork quilt of laws, a Congress that's struggling to make sense of it.

[00:21:17] And various states, they're like, we can't we can't wait that long to protect our folks from the use of this technology. And it's powerful. The amount of the amount of hidden bias, the amount of inference it can make through pattern matching across a data set that even the best of us can never hope to keep in our heads. It's dramatic. And so I respect their attempt to to try to bring some kind of sanity and some kind of security to it so that we can all benefit. Right.

[00:21:46] This is a positive. This is a positive use of AI, but it's a lot of work. It really is. I did another layer of complexity with the EU product liability directive. And on that side of things, a question I've got to ask you here is if if an AI coding assistant may be introduced a vulnerability that later leads to a breach or regulatory failure, where does the liability realistically sit in this kind of situation? Yeah, I mean, that liability remains with the organization deploying the code.

[00:22:16] This is the this is the companion piece to that AI act. You know, in some ways, it's even the sharper commercial risk. AI act tells you, here's what I'm going to build. Here's how I'm going to govern it. And this PLD tells you exactly what happens when something goes awry. You know, you're now I'm going to say this before I say that. Like we've talked for years about how software is a factory. Right. Agile was a great, you know, I mean, not that I necessarily subscribe to everything.

[00:22:44] And Kent Beck and folks like put to but agile was really turning into the factory. What's that pipeline? Right. And then you got folks like Gene Kim, who came in with this brilliant book, The Phoenix Project, you know, which was which was a lot of that theory of constraints that Eli Goldratt put out there. And this turns turns, you know, this law explicitly says, essentially, if you're producing software, you are now a manufacturer in the legal sense.

[00:23:10] And the legal protections that software companies historically enjoyed, the ambiguity about whether software was really even a product, a high burden on the plaintiffs, limited damages. Those have all been systemically dismantled. So any failures here point to gaps in governance, testing and oversight, not just tool usage. We get a lot of questions around due diligence then. Right. And so you want to go in your organization and look at were you healthy about are these tools approved? Are they understood? Are they used within clear guardrails even? Right.

[00:23:39] It doesn't shift responsibility at all. In fact, if anything, it increases the need for strong controls. I think a lot of us and, you know, in engineering, a lot of us who have grown up with this technology forward bent of always just being a builder, that's frustrating for us because it feels like we're pulling the brakes on something that's intended to help us go faster. Right.

[00:24:07] It's almost like you're getting in your sports car, you know, and flooring it. And the guy in the in I was about to say the right seat, but the left seat in your case, right, still got his hand on the parking brake. Yeah, it certainly feels that way sometimes. And then if we also look at the traditional IT governance models that we were both seen since the start of our careers, they were designed for software built and deployed by human teams.

[00:24:33] But those models are obviously going to struggle when AI systems and AI generated code become part of the development workflow, too. Right. What are you seeing here? You know, we we ran a really interesting experiment the other day. One of my teams, I'm going to give one of my wonderful engineering managers, Carmen Kwan, a shout out for doing it. She was amazing. She ran a but we called it bug day. Yeah.

[00:25:02] And so it's a bug bash and you've done these forever. But we did it all with tools and we did it all without the engineers actually writing the code, but prompting the tools with a lot of human oversight. And what we we discovered, it wasn't surprising to me, but it just reinforced it, which is all of these processes are designed for deterministic human driven systems where the risks are visible and contained. Yeah. And the bottleneck, again, just shifted. It became reviewing and oversight of these systems.

[00:25:31] So we've always placed the focus on things like pre-deployment checks, which assumes human ownership of decisions. And all of a sudden, AI is introducing non-deterministic behavior and logic within workflows, often without clear ownership. In fact, the reason we like AI, the reason we enjoy interacting with it is it's somewhat random. You know, there's there's a temperature baked into a model, which is exactly how random this thing is. And it's, you know, I say random because, you know, this is math, right?

[00:26:00] It's a bunch of matrix multiplies. But, you know, some people would say the creativity, which I think I don't like to anthropomorphize AI personally, but I get it. Right. We like that as a human. That's that's the part of being human. You know, all of us who've been in this industry for a long time probably went into it because we like determinism. You know, the machine does exactly what I said it did unless somebody else broke it and put a bug in.

[00:26:26] And I think, you know, part of, you know, part of our business in LaunchDarkly with these runtime controls and checks is about bringing predictability to a determined and determinism to a probabilistic system by putting guardrails in place that create. I mean, I'm going to say this. I'm not going to create stochastic processes that are repeatable by nudging things back on course or course correcting along the way. So it takes that creativity, leaves it there and helps bring it back to steer to the outcomes you want.

[00:26:54] You know, this requires ongoing real time control. So monitoring, guardrails, rollback, kill switches, et cetera, and not just approval processes in there. It's not about saying, yes, this is good to go. In fact, I can get another LLM to do that. And then what do I need to be around for? But I want to be able to shut this thing off when it goes haywire. And it doesn't necessarily even go haywire, Neil, because, you know, I made a logic error or the AI made a logic error.

[00:27:22] Sometimes it goes viral and all of a sudden I'm burning tokens because I don't know if you saw the Chipotle meme last week where they were used. Last week, week before, someone discovered that the Chipotle chatbot could write Python for you instead of making your burrito bowl. So it was super interesting. I'm like, why pay for a Claude subscription when I could get Chipotle to do it for me? That's genius. Yeah, I mean, it was never intentional, but you'd want to shut that off in 200 milliseconds or last, please.

[00:27:53] So, you know, governance, the point that I'm getting at, though, is governance has to evolve, right? It's no longer static approval, but it changes into continuous control embedded in your delivery systems. And I think the best organizations did this in the move to the cloud.

[00:28:11] They went from static audits and static remediations to things like, you know, in an AWS ecosystem, for instance, looking at CloudTrail CloudWatch, putting Lambdas in to remediate security issues in real time. And so your check was that this Lambda fire when I, you know, unencrypted an S3 bucket and force it back to encryption. Instead of checking, are all my buckets encrypted? I think AI is the same way.

[00:28:36] And so the best organizations that are going to function and make use of this technology in a safe way have those automated remediations and automated feature flags and automated kill switches. Internally here, like we're doing, you know, a project internally where all of our PRs are automatically examined by an LLM. They're wrapped in feature flags before they go into production, right? All of there's an there's observability data created as part of this process. So that's a huge augmentation.

[00:29:04] The things we would teach a developer how to go from a computer scientist to a software engineer, right? Because the two are very different, in my opinion. Moving from computer scientist to software engineer is learning about how do I do day two? Day one, they teach in school. How do I write code? Day two is how do I actually make this work for, you know, the 54 trillion flag evaluations that we're doing every single day across the planet? And that's a learning curve.

[00:29:29] And now we've been able to help augment that with the use of these tools to say these are the behaviors that we codify in our organization that we've learned through years of operating experience. This is going to help teach you how to do that or do it for you on day one. And I always try and give everyone listening a few valuable takeaways.

[00:29:46] So from a technical perspective, how can somebody listening in their organization better increase visibility into how those AI tools are being used inside live systems so they can maintain that element of control, but doing so without slowing innovation? And I understand it is a notoriously tricky balance, but any tips or advice on how to do that? Yeah. AI moves faster than human scale.

[00:30:16] This is scary to a lot of people because, you know, I've never been on a hamster wheel and run this fast in my life and felt like I wasn't moving around the cage. Like, we're in an arms race and I don't use that term lightly. I'm very, very protective of it, but I think this is a true arms race that you're not going to be able to get out of.

[00:30:41] And so the way that you win that is you make AI usage visible by design across the development lifecycle. You know, and I think there are a lot of small companies out there trying to get audit trails in place for their usage where it's not just the outcomes, right? We get every time it checks into GitHub, like I get a history of what went in there. But it's understanding the why of how those decisions were made.

[00:31:06] You have to embed governance and controls directly into the delivery workflow. So all of these AI generated changes are tracked and reviewed and the attribution to the author has to happen in there. And it's nice. Like, I'm one of one of my projects here right now is what is our internal development platform look like in a year? How do we enable developers? So this stuff is baked in there so they don't have to think about it. They just have to do.

[00:31:34] You need to have incredibly good runtime observability for this to work. You have to monitor how these AI influence features behave in production. And in light of, you know, our EU AI laws that we were talking about a minute ago, like that's even more important that you use techniques and software packages to monitor how these are behaving in production.

[00:31:55] So that when it comes down to defending yourself, you have a commercially reasonable answer to, I did what was, I tried to do the right thing here. And if we broke it, we can correct it. But we did everything in our power to figure out that it was the right answer. You need guardrails as well. Approval layers, policy enforcement, restrictions on sensitive data. So your internal data classifications and governance become even more important.

[00:32:23] And especially when you're running, you know, a data lakes or, you know, modern companies don't even run data lakes anymore, right? We run data fabrics where data is a product that your individual departments or services or products actually just vend internally. Now you have a distributed governance project or problem to deal with. And so how do you classify, protect that, put the guardrails in place? So, you know, my LLMs, my agents here. Every morning I run agents. I'm sure you do too, Neil, at this point, right?

[00:32:52] My agents come through my emails for me. They go through my daily schedule. They do all of this. And they have some pretty privileged access. It's not like, let's be clear, I'm not an open claw, like just YOLO the world, right? You can have my checkbook, my Amazon account. Please have it all. No, but in a more serious way, though, it does have access to some of the things with the minimum spanning set of permissions I could come up with. But it still needs some sorts of privileged access. I just have audit trails on it.

[00:33:20] And we want to make sure that there's policy enforcement restrictions. You know, there's a governance process. So as I build these, as my engineers build these, a lot of them are skills internally. And we review them together. And then we share them so that you don't have to have somebody else take another attempt at rewriting it as easy as it is and maybe get the security permissions wrong. So and people are grateful for that. We're sharing best practices. We're sharing research tools. We're sharing all of our internal operating agents.

[00:33:46] And there's, you know, a couple hundred to a thousand already running around here. They're all, they're governed, right? There's a check-in and audit trail and people are constantly looking at them. And then the last bit of this is you've got to ensure that there's rollback mechanisms and kill switches, runtime kill switches, flags around that to defend yourself and really gain control when issues arise. And those issues, you know, could be costing. They could be fairness.

[00:34:13] They could be the jailbreaking of coding Python instead of making burritos. You know, you want to make sure that as you have probabilistic behavior that is, I mean, non-deterministic behavior, really, that if you don't like how it's behaving, you shut it off. It's like in some cases, it's like, you know, taking your toddler out of the classroom and giving them a timeout once in a while. But you got to be able to reach in there and do it. Hopefully we do it before they're all sentient, right? I hope so.

[00:34:44] Don't shut me off. I'm sorry, Dave. I can't do that. I was going to say, getting flashbacks of hell in 2001 there. And finally, before I let you go, for boards, CISOs, DevOps and other leaders that are listening today around the world, what questions should they be asking right now about AI governance, accountability, et cetera, to avoid security, compliance and liability exposure in the years ahead?

[00:35:11] Because right now everyone's excited about the new tools, et cetera. But the so-called boring stuff, the important things that could bite you later on is very real, that threat is. So any advice there? Yeah. To me, are they asking me or are they going to ask Claude? Yeah. True. I'm kidding on that one. Look, it's a really good question. This comes down, this comes down in my not so humble opinion to the difference between wisdom and knowledge.

[00:35:41] Yeah, yeah. Which is, you need to have the experience and the wisdom to really raise this at the senior level and say, number one, do I have visibility into where AI is being used across our teams? And I think if you spend some time and just ideate on this, be curious about it, ask the questions, you're going to find it's used everywhere. Everybody's got the, I don't know, what are we calling it? It's not the, it's the little star, right?

[00:36:10] That they, I forget what we, what are we calling in the industry now? I've, I've even forgotten the AI, the AI token in the corner there that just blips. Everything has it now. So you say, well, you know, I'm not using it to write my, you know, spreadsheet or whatever. Oh yeah, you are. It's absolutely there. So I think you'll be surprised. I think the next question you want to ask is who is ultimately accountable for the AI generated outputs? And how do you, how do you create the attribution back to, to that person?

[00:36:40] Uh, the next question is, can we trace and explain those decisions or code if we need, if we needed? Right. And I know our, our, you know, chief legal officers definitely want to see that, but even as an engineering manager, I want to do that because it helps me make the architectural decisions about how my product looks and grows. And it helps make the business decisions of these are the two use cases and we have to pick a path. The next question is, of course, do we have runtime controls like rollback or kill switches in place? Um, that's absolutely critical to be able to make split deck second decisions.

[00:37:11] Uh, code still takes time to ship. So putting it out there, enabling it, uh, in production with runtime controls is all the more valuable. You can't just ship forward or roll back. You have to have a runtime kill switch in place. Are my teams using AI within the approved guardrails, especially with the sensitive data and how am I protecting that data? Um, you know, we're, we're doing all of the typical endpoint threat detection and response on our, on our machines here. Like you would, you would do.

[00:37:39] And we're watching who's connecting and how they're connecting to some of these services. Uh, and we do it not because I don't want people to use it, but because I want them to use it governed through our corporate, you know, our corporate account with, with Anthropic or open AI or Gemini or any of the tools you want versus the personal account that people were so eager to get started with. Um, and the benefit is I'll pay for your tokens internally when you do it, right? Which it's not just stuff in cheap.

[00:38:06] Uh, the next question of course is how are we evolving our culture and bringing people with us to ensure that AI is being applied responsibly, fairly, and with the right checks in place. Uh, and we're, you know, we're highlighting internally all the great projects people are doing. We're doing these partnerships and training. I want everybody to come along. Like this is an inevitability. You can't dig your heels in and be a Luddite for it.

[00:38:30] Um, whether you want to or not, as much as I'd love to love to go back to the days of, you know, walking out of the office at five o'clock and shipping on a, on a floppy disk. Uh, it's not going to happen, right? We're 24 by seven. I mean, and now it's, it's faster. So, you know, can we bring people with us into that reality in a way that's, that's comforting to them. Uh, and in a way that helps them feel empowered to be better with these tools and grow.

[00:38:56] And, and when they do, I mean, I'm impressed by every single engineer in the organization that's shown me something using these tools where I'm like, that's really clever. Or I'm copying that, or I'm posting about that, or, you know, and it's great to see. Um, and then the last question I have is, are we spending enough tokens on the generative phase? Are we spent, you know, versus inference? And how are we controlling the costs around this? Uh, I think that's a huge question. Um, I think AI is, is still in the subsidy phase.

[00:39:25] Uh, and so there'll be a reckoning where this stuff gets a lot more expensive, uh, in the future. You know, and we're seeing this through rising energy costs, uh, a lot of arguments, at least here stateside about who pays for electricity, um, hardware costs, you know, and while we still have processor power and inference power accelerating, right? It's not free in the, in the rare earth minerals that we mined for this, uh, in the raw resources, the land, the building, the things that still have to happen to make it possible.

[00:39:54] Um, I monitor token spend. I monitor AI based check-ins, but to me, they're lagging indicators of adoption. They're not metrics to optimize, right? I don't want to ever tell my developer you to spend two times your salary in, in tokens, please. Wrong incentives, right? That's, that's a perverse set of incentives for somebody. Just like 80% of my check-ins need to be AI. Great. I check in one line at a time and now my number goes up. Like that's not the right answer, right?

[00:40:23] The right answer is measuring the outcomes and, and measuring the backlog. Um, I'm going to say, I don't look at backlogs anymore. Um, you know, we used to say, okay, what's the backlog for this team of engineers, right? It's three months, six months, nine months. It's changed. An engineer can supervise agent teams. Um, HBR tells me it's about four is the optimum number of agents to supervise at any given point in time. Any more than that, your context switching kind of destroys your, your productivity. So my question to my engineers is, are you managing those four agent teams?

[00:40:53] If not, why not four, right? And then is the project that's on the backlog, uh, something that has a positive ROI where the token spend for the outcome, um, and the time and the context switching has a positive return for our business some way or another. And that's all that matters anymore, uh, because I can do things that, that I never could do before. Cause I wouldn't want to invest 18 months of engineering time to do it. Um, we had an 18 month project here, for instance, uh, we did an eight weeks or we're finished.

[00:41:22] We're just finishing and it's going to be eight weeks for something that would have been 18 months prior to this. And I love it. And the token spends not cheap. Like I won't tell you the numbers, but it's a lot cheaper than 18 months of engineering time that would have done it. And I love that. I love that change because now I've gone, I said I was nerdy. I'm not right. I was always been a theoretician. Like I love the theoretical. I love the proofs. I love, you know, spending my time with the journals, reading the papers.

[00:41:48] I have to be an experimentalist now because instead of spending a lot of time on theory, we just try it and we can measure it really quickly. We've got runtime controls to shut it off. And, you know, if I throw it away, okay, I spent, you know, a few thousand dollars in tokens to build it. Fine. Right. So, so change your mindset into being an experimentalist and forging that path forward by doing. And I think that's the fundamental shift of what questions they could be asking. Are you experimenting? I think when it comes down to it at the end of the day, Neil.

[00:42:18] Wow. So much to take away and think about that. I'd love to invite everyone listening to share your thoughts on this, your insights, your experiences. So for everyone listening, I will post a link to the launch darkly report that you referenced, I think, earlier in the conversation. But anywhere else you'd like me to point everyone if they want to connect with you, your team or just some of the work and blogs and everything you write, where would you like me to point everyone? So I do tend to post on LinkedIn fairly frequently.

[00:43:14] Definitely. I do tend to post on LinkedIn. I will add links to everything that you mentioned there. And as I said, I invite everyone listening and watching to post your feedback, especially around shadow AI in DevOps when AI written code could cause a breach. Who is responsible? This is something we could debate about for hours. But more than anything, just thank you for starting this conversation today. Thanks, Neil, for the time. This has been great.

[00:43:45] There was so much to take away from this conversation today. But for me, I think one of the biggest takeaways was simple. AI, yep, it's speeding up software delivery. But sorry to be the guy bringing the bad news here. It doesn't remove responsibility. If anything, it's raising the stakes. And yet shadow AI might feel harmless in the moment.

[00:44:08] A quick shortcut here, a productivity boost there, a faster way to get the code out the door on a Friday afternoon. But without visibility, controls and clear ownership, all of these shortcuts can get expensive very quickly. And what I loved about Cameron's perspective here is that he did not come at this from a place of fear. He came at it from experience. And the answer was very clear.

[00:44:35] It's not a time to panic or block progress or stifle innovation. It's simply a time to put the right guardrails in place so teams, yes, can experiment to their heart's content. They can ship code and still remain in control. So as AI becomes part of the everyday developer workflow, I think the question is no longer whether teams will use it. The genie is out of the bottle now, isn't it?

[00:45:01] I think the bigger question is whether your organization is ready for what comes next. And how are you managing this transition? As always, techtalksnetwork.com. You'll find 4,000 interviews, lots of ways of getting in touch with me there. And please, feedback. I'd love to hear from you. A quick thank you to NordLayer for supporting the podcast and helping me make these daily conversations possible.

[00:45:29] And if you are listening and you're responsible for security or IT, you will know the reality. The reality that most of your risk now sits inside SaaS apps and browser activity. That gap is exactly what NordLayer is addressing with its new business browser. So instead of bolting security on from the outside, it builds it directly into the browser itself.

[00:45:55] This means you can control access, monitor activity, enforce policies and reduce shadow IT all from one single place. And most importantly, it does it without adding deployment headaches or complex onboarding. You get things like browser-based data loss prevention, SaaS access control and zero trust browsing, but delivered in a way that your team can actually use.

[00:46:21] So if you've been trying to simplify your stack while improving visibility, please check it out at nordlayer.com slash browser. But that's it for today. So time for me to check out. I'll be back again very soon with another guest. And hopefully you will meet me here. Same time, same place. Bye for now.