Securing AI-Generated Code Without Losing Speed With Checkmarx

[00:00:00] The leading issue of agentic AI in businesses right now is ensuring agents act with compliance guidelines. And Denodo applies guardrails across your entire data estate. By aligning your company's data infrastructure under one system, these guardrails perform consistently across your platform. So start scaling your business and start with Denodo. Simply visit denodo.com to learn more.

[00:00:27] How do you know whether code generated by AI is ready for production rather than working in a demo? Well, AI can help people build software quickly, but reviewing what it produces still requires an understanding of design, context and security. And as output grows, development and security teams need to keep up

[00:00:54] without turning every release into an approval process. So my guest today is Frank Emery. He's the VP of Product over at Checkmarx. And we're going to discuss the skills and controls behind that very challenge. We'll talk about why understanding generated code matters, where responsibility sits when vulnerabilities reach a product, and which changes still need human review.

[00:01:23] And Frank will also explain a more layered approach to security testing in everything from code generation through deployment and the measures teams can use to assess whether they are actually reducing exposure. So if you're a developer or business leader, the conversation today will offer questions about building software faster while most importantly retaining confidence in the result. But with that scene set, let me introduce you to Frank right now.

[00:01:54] So a massive thank you for joining me on the podcast today. A warm welcome. Can you tell everyone listening a little about who you are and what you do? Definitely. So my name is Frank Emery. I'm the VP of Product here at Checkmarx, overseeing our platform and our engines. So a good chunk of all things that Checkmarx works on. And I joined Checkmarx about a year and a half ago. So it's been an interesting year and a half.

[00:02:21] I would say joining a security company right before Mythos came out and right before the entire industry changed has been a really interesting experience. And I started my career 15 years ago counterintuitively working in finance. It's what I went to school for, but did not stick around in. But I've been working in product ever since then. Starting out after going to school for finance, learning about agile and being an agile coach before switching into product

[00:02:48] for a couple of years and really focusing on startups and database startups, really, for most of my career before moving into security. So I've been working with technical customers and technical teams my entire life. But definitely things have been, as I said, interesting the last year and a half with all the changes that we've been seeing in the space. Yeah. The pace of technological change is phenomenal at the moment. And over the last few years, there's been a lot of talk of Gen Z

[00:03:16] and the first generation to enter the workforce before. Well, they cannot remember a time before the iPhone and the iPad. Of course, iPhone was 2007, iPad 2010. But we're now moving on that conversation to the first generation of developers who have learned to code alongside AI and they're entering the workforce. So I've got to ask, from what you're seeing, what skills have become more valuable now and which traditional skills maybe are at the risk of being neglected from what you're seeing? Yeah.

[00:03:46] The way I've always thought about this is I like to use the system one versus system two thinking. And when I've seen AI come along, what I've really seen it primarily start with, especially with the younger generations, is it's replaced sort of the system one thinking that we've all been doing. So if you think of coding, these are sort of the day-to-day tasks that they're not particularly difficult, but they do take up the bulk of time during the days.

[00:04:12] So this is like building websites, basic implementation of code, not really the architecture side of it. And that's what I've seen AI take over for first, which is sort of the traditional skill set that has then led to be neglected. And so what you have is a bunch of folks that are entering the industry that never really got a chance to learn how to do the basic system one stuff. But the problem is, if you don't learn that basic stuff up front,

[00:04:41] then you never get the ability to build out the really valuable skill sets, which is your system twos, which is your architecture, your planning, your thinking, all of the really complex stuff that is now more valuable than ever. And that's sort of the change that I've seen. So nowadays, people will say, hey, AI, go build me a website. And AI will do that. And it's doing all that easy stuff for you. But then you never get the ability or the experience of going back and saying, okay, well, did it build the right thing?

[00:05:09] How can I review this to make sure that what's built is going to work in production? I was talking to some classmates of mine last year, and they were all like building these applications. And so for them, coding used to be this really difficult skill to learn. Now coding is just speaking. And they were all saying like, yeah, I built this amazing startup in an afternoon. Like, you work in security? How can I take this production? And then I'd ask them, okay, well, cool.

[00:05:36] Did you like review your code? What's the architecture look like behind it? Like, have you tested it yet? And they all said like, what's architecture? What do you mean review? Security? Well, no one's going to break into my house. And it kind of dawned on me that like, you have all these like super empowered, empowered, like bright people that are now, they can take on these new tasks and build these new companies.

[00:06:04] And they literally have no idea about all of this other stuff, which is unfortunate because if you do have that skill set now, you're like the, you're what everyone wants to talk to. So if you do understand how to do architecture reviews, security reviews, you know, look at an application and understand like, is it doing the right thing? Is it going to work properly? Like that, that's where all of the value is now. Yeah. It feels like we've come full circle and gone straight back to, let's move fast and break things again, you know? And of course, what we're talking about here is AI assisted developers.

[00:06:34] They can commit code three to four times faster than their peers. So what happens when that output grows faster than security teams can realistically review it? Or are those security teams using AI too? Yeah, it's, I would say, yes, security teams are using AI, but to the first part of the question is, you know, what happens when you commit that fast is you accumulate issues. So, you know, I know in some circles they talk about AI contributing to management debt

[00:07:03] or compounding where decisions compound on decisions. It's the same thing that happens with code. So if you're developing four times as fast, unless you have really, really good checks and balances throughout the development process, then you have risk compounding on risk. Vulnerabilities accumulate on vulnerabilities. And unfortunately, a lot of the guardrails that exist within coding assistance today, they don't necessarily scan everything.

[00:07:30] They're not necessarily as comprehensive as they could be. And so if you imagine, say, every time you make a commit, you know, the systems that in place catch 20% of your vulnerabilities. Well, if you're committing four times as fast now, then each time you make a commit, that 80% you're missing grows and grows. And unfortunately, if there's nothing there to catch everything, once it's made into production, you end up with a fairly large backlog of risk that exists.

[00:07:58] And in fact, what we see in our Future of AppSec report is only about 9% of organizations fix 90% of their vulnerabilities. And even 90% isn't everything. If there's a critical vulnerability in that 10% you didn't scan, you still get hacked. There's actually, you only need one vulnerability to be hacked. And so this accumulation means that backlogs are growing more and more. And though there are tools in the security space to help organizations catch up, you really need to make sure that whatever tool you're using

[00:08:26] is able to find as much as possible because of the compounding code commits that are happening. Yeah. And anyone that's worked in a technical team will know all about that accountability problem. And when something goes wrong, the developer, the deployer, the vendor, the network guy, everyone's pointing at each other at that moment, everything goes wrong like that Spider-Man meme, you know. But if AI generates vulnerable code and it does pass through several tools and teams, which it can do, especially as it gets more smarter,

[00:08:56] who should own the risk when something goes wrong? Who's going to be the person accountable for that? Ultimately, it's the organization. So at the end of the day, if you're the end user of a product and that product gets breached or hacked, you know, you're not necessarily going to know the individual team that dropped the ball, but you are going to know the organization that dropped the ball. And so I think ultimately, you know, the buck stops with the company that's putting products out. I think within an organization,

[00:09:24] if you look at the individual team levels, it's going to come down to, if you're having an AI write your code, you know, you're responsible for that AI's writing of your code and that all bubbles up. And so the individual developer, they're accountable for validating that whatever they wrote is secure. Sure. And that's, you know, to the original point about why that skill set of validation review is so important now. And then if you're a team leader, you're responsible for making sure that every developer on your team is validating their changes.

[00:09:53] And if you're running an AppSec organization or an AppSec program, you're responsible for making sure that all those team leads are adhering to application security standards. And so it does all ultimately bubble up, but that is still a lot of points of failure along the way, where as people speed up and as they lose the ability over time to do, you know, reviews, security, architecture, all of that just increases risk across time. And unfortunately, AI is making it a lot easier to exploit things, but at the same time,

[00:10:23] it is helping both with detection remediation as well. So it's not all bad. There are ways to fix this. And we're seeing a lot of organizations starting to actually use AI to help with detection remediation to keep up pace with vulnerability creation as well. And if we do have a young Gen Z developer who's entering the workplace for the first time, maybe even going to an interview and wanting to make the employer understand that they know what they're talking about and what they're responsible for, what should they be expected to understand

[00:10:53] about reasoning, validation, secure design when they're orchestrating AI generated output rather than writing every single line themselves like it used to be done? I think first and foremost, they need to understand what the code being generated does. I have seen far too many instances of people firing off a prompt and getting something back and immediately executing a pull request. So they're going right from prompts to deployment. And when you do that, you're not necessarily understanding

[00:11:23] what that code is doing. And a lot of the times, vulnerabilities aren't necessarily just a cross-site scripting vulnerability. They're more logical. They have more to do with what that code does in context. And so I think if you're a new grad, if you're new in the industry, I would say don't just trust what AI is doing blindly. You have to understand what that code is meant to do, how it fits into the overall design of your application. And I think you need to have a good idea

[00:11:52] of what secure design looks like. We see a lot of blogs online that will talk about, hey, AI passed this functional test. The AI made code and it did what it's supposed to do. Like if you press this button that AI made, that button will do what you expected the button to do. But there's a big difference between being functionally correct and being secure. And I think that's where people really need to do research and figure out and understand

[00:12:21] what that difference is and how to make the code that's being developed by AI not just functionally correct, but secure as well. And that is the Young Developer Lab, an organizational level. How can they decide which changes an AI coding tool should make independently while others will still require that human review? It's a good question. We actually talk to a lot of customers about this. And I would say that

[00:12:51] what you let AI do independently will depend a lot on how your organization has implemented AI across the board, how mature you are. I have talked to some organizations that have entire teams that are just AI. There's one company I've talked to. They have a team where it's an AI that defines its own requirements, writes its own code, implements that own code and pushes it into production. That's one end

[00:13:18] of the extremely mature spectrum. And then you have the other end where organizations aren't even scanning their code for security risks, AI generated or not. And so I think when you look at what should AI be allowed to do independently or not, you have to start with what control plans exist within your organization so that when AI takes actions, there's something in place to make sure the actions it's taking are correct and secure.

[00:13:48] And so this could be anything from, you know, if you have automated scanning and automated remediation in place, then great. You can probably let AI do a lot more than it would in other situations. If you don't have any of that in place or if your scanning and radiation really aren't up to where they need to be, then you probably have to have a human be more involved. And so I think that's sort of one way to look at it. The other way to look at it is sort of the idea of routine

[00:14:15] versus more advanced work. I don't think AI is at the point yet where I would let it go in and define architecture for my business overall. I think it can certainly help in the process, but I think that still needs human review. But low risk changes, I'm even seeing people in product do this now where I remember back in the good old days if you got something built and you looked at the demo and you saw the colors were wrong, and you'd have to cut a ticket and it would take a week to change colors. That is low risk,

[00:14:45] that's easy. Automate that or have non-technical audiences do that, it's pretty easy. The architecture side of it is much more risky, it's much more critical to a business and that's where I'd still have humans be in the loop. And then the other part that's kind of interesting is around testing. We do see AI coming up more in security testing, but what we see a lot of organizations doing is adopting a policy of separation of church and state where the AI that's doing the development

[00:15:14] can't be the AI that's also doing the security testing because they'll cheat. So, you know, using AI as part of the security testing process as well, but making sure that it's a different AI process that's part of that so that you can ensure that, you know, you're mitigating risk. And at a time where many leaders are feeling the pressure to move forward more quickly, security controls can be seen as something that become a late stage gate that actually slows down delivery. So where should checks sit

[00:15:43] so teams can reduce application risk without losing that speed that AI provides? I mean, the easy answer is as early as possible, but I would say it's easier said than done. What we've seen is everything from people doing security checks as you write, so validating as you type out code that what you've typed is correct to even people integrating security directly into prompts. So for example, if you're using, say, Cloud,

[00:16:13] having security integrated directly within the Cloud that says, hey, generate a secure prompt instead of an insecure prompt. So you can start very early in the process. And my recommendation is always use sort of a Swiss cheese approach. So, you know, if you say make it so that you're doing a rigorous scan at code typing level, then you're never going to get work done. It's too time consuming. But there's easy stuff and fast stuff that you can do at code generation time. So do it there. And then as the code is committed, then bring in scanning. And as that code

[00:16:43] moves into applications, then look at application testing. And it becomes an exercise of understanding what is the appropriate level of fidelity and time investment in scanning for this stage of your development process. And so we look at the SDLC from everything from code generation to your applications that are live and production. And there are things you can do at each individual stage that catch individual things and have a cost, both in terms of time and complexity,

[00:17:13] appropriate for that stage. And so our recommendation is always do what's appropriate at that individual stage at that stage in order to ensure you're being secured. To be very specific, like I said with the earlier example, at code generation, we use something called DevAssist. That's something that Checkmarx offers that validates your code as being generated correctly. And then once that code goes into PR, we typically see people do static code or supply chain scanning to make sure that the code didn't create

[00:17:42] any vulnerabilities or introduce any risk within your organization. And then once that code is pushed through a pull request, we typically see people do more application testing. So whether it's an AI pen testing or dynamic application testing that's more time consuming, but will catch more stuff. And then once it's deployed and fully in a runtime environment, runtime testing to make sure that while it's out in the wild, it's being used securely. So each of those is sort of a purpose-built tool for each of the steps of development. And when it's integrated correctly,

[00:18:11] it all works seamlessly and quickly and people are usually pretty ignorant to it running in the background unless you're introducing vulnerabilities, in which case you'll be told pretty quickly that what you're about to push has problems. Yeah. And there's also a pressure to continuously evolve and adapt as you go through any process now. And validation frameworks and security standards, how should they be changing and evolving when generated code can be produced at scale that no human team has got a chance

[00:18:40] of inspecting line by line? Any tips and advice on that side of things? Yeah. I think the big thing is that it has to switch to more of a continuous validation framework. In the past, it was much more of a, I'd say like a stage gate for lack of a better word. And that just doesn't keep up. And so with a continuous approach, you can basically work as quickly as automation can work. And automation is going to work as quickly as AI generated code can. So I think that's the first step

[00:19:08] is that continuous approach. However, it's still important to have those checks and balances, but having them at an appropriate step. And so for example, we talked to a lot of folks who are breaking their validation frameworks into sort of a two-step process. So like step one is continuous validation as code is being generated. Step two is, on a daily or weekly or monthly basis, having more of an AI-driven analysis to seed anything like escape

[00:19:37] or not get caught by the continuous scanning. And then once you have those sort of two views, taking a very exploitability-based approach to prioritization, which we see a lot more in the market nowadays, of don't just look at vulnerabilities, but look at exploits and what vulnerabilities you're contributing to those to help you prioritize. And if you adopt those three practices, and many of those are new, then it does kind of help you scale up

[00:20:05] beyond the human limitations that exist today. And of course, so much has changed and yet so much stays the same. And the old IT mantra of you can only improve what you measure, that's never been more important than it is right now. But knowing what to measure, that is often a stumbling point too. So what measures are you seeing that show AI-assisted development is actually improving software quality, reducing real risk rather than just simply increasing the volume? It's a good question. So AI-assisted development

[00:20:35] is genuinely improving security outcomes when teams are moving faster. It is, like I said, helping quite a bit. I think the big thing is we do see vulnerability exposure windows are shrinking. However, we're also seeing remediation effectiveness improving. And so when we look at specific measures, I think it comes down to time to discovery is a big one. I think the volume of what's discovered, so we talk a lot about fidelity or F1 score is another thing

[00:21:04] that we measure quite a bit. And then I would say time to remediate. is another big one. So those three areas. And AI-assisted development is improving things across the board. It's interesting, like if we look a year, year and a half ago at what products existed in the market sort of before AI entered security versus now, we're seeing like marked improvements. Fidelity is through the roof better. AI is helping us find more and faster. We're seeing remediation windows or remediation periods

[00:21:33] now drastically improving alongside the exploitability windows, which is fantastic. So things are being exploited faster, but they're also being fixed a lot faster. And so those are some of the measures that we look at and they are all genuinely improving, which is good to see. It really is. And for people listening, if any of our conversation resonated with you today, the good news is you're not alone on this journey. Checkmarx, for example, helping the world's largest enterprises get ahead of the application risk that we're talking about without slowing down

[00:22:03] development. So for anyone listening that would like to carry on that conversation, where can they find out more about everything we've talked about today? Yeah, the best place is our website, checkmarx.com. We have a lot of information about Agentic application security testing as well as events we're going to, webinars we hold, lots of goodness there. Awesome. So much great things in what you've raised today and great work as well. I really support what you're doing here. And Frank, I will include links to everything everyone's

[00:22:32] mentioned that you've mentioned and for people listening, please go check it out. Continue the conversation, but thanks for bringing it all to life today, Frank, in a language everyone can understand and regardless of age to hopefully people find value in this, but thanks again. Absolutely. I'm happy to be here. I think Frank's point about code being functionally correct while remaining insecure is a reminder for anyone assessing it, producing an application and understanding it, whether it is ready for production, all involve different responsibilities.

[00:23:02] Today's conversation, I think highlights why security checks need to fit the work that is being done. A quick check during co-generation will serve a very different purpose from testing later in the process because teams need those layers to work together and with clear ownership when something needs attention. All incredibly important. And for leaders focusing on measuring progress, the discussion, I think, returns to

[00:23:31] discovery time, detection quality and remediation because more output alone cannot establish whether the business has reduced its exposure or not. So a big thank you to Frank from Checkmarx for joining me here on Tech Talks Daily today. You'll find the company website, report, resources, everything you need in the episode notes over at techtalksnetwork.com. But over to you, how is your team reviewing AI-generated code?

[00:24:01] Where are you finding gaps appearing? I want to hear your thoughts, experiences and everything. So please, let me know. But that is it for today. Thanks for listening as always. I'll speak with you all again tomorrow bright and early. Speak with you then. Bye for now. Bye for now. Bye for now. Bye for now. Bye for now.