How Moderne Is Scaling AI for Code Refactoring in the Enterprise

[00:00:04] - [Speaker 0]
Welcome to AI at Work, a podcast which is part of the Tech Talks Network. And in this podcast, we're gonna venture into the transformative influence of artificial intelligence inside the workplace. And our discussions will focus on both the remarkable breakthroughs, but also the complex challenges of integrating AI into our everyday business functions and workflows. And, yes, we will do our best to avoid hype. Because this year, it was revealed that there is a big ROI problem with many AI projects in organizations of all sizes.

[00:00:44] - [Speaker 0]
While AI assistants have proven their worth in helping individual developers optimize and write code, scaling that innovation across an organization's entire code base, well, that requires a fundamentally different approach and something we don't talk about enough. And Justine's expertise in AI driven code refactoring and her book AI for mass scale code refactoring and analysis released last September makes her the perfect guest to explore this topic. So today, we'll examine how AI can streamline the process of large scale migrations, reduce technical debt, and empower engineering teams to work more efficiently. And as businesses continue modernizing their applications, I wanna ask questions such as what will AI play in ensuring code quality and adaptability, but on a massive scale? How do we get there?

[00:01:40] - [Speaker 0]
And how can organizations leverage some of these tools to manage the complexity of enterprise level development? Well, let's get today's guest up to find out more. So a massive warm welcome to the show, Justine. Can you tell everyone listening a little about who you are and what you do?

[00:01:57] - [Speaker 1]
Absolutely. First of all, thank you for having me. So I'm Justine. I am based out of Montreal. I've been working at Moderna.

[00:02:04] - [Speaker 1]
Great company that uses and maintains and developed and created Open Rewrite. And I work as an AI research engineer. So I do everything that has to do with AI, but I also do a little bit of research in that space as well to make sure we always have the best technology to offer to our customers. For some context, at Moderna, we help companies keep their systems running smoothly by automating updates to thousands of applications or a million lines of code.

[00:02:33] - [Speaker 0]
That's the coolest title in the world, isn't it? Especially right now, AI research engineer. Does your LinkedIn get bombarded? I'm curious.

[00:02:40] - [Speaker 1]
Absolutely. It does. I definitely do get a lot of messages. Either people wanted me to use their platforms or some people come with questions, which I actually like getting questions. I think it's always fun to help out people.

[00:02:52] - [Speaker 1]
So anyone who wants to ask a question about AI, they should definitely keep messaging me. I love answering those.

[00:02:58] - [Speaker 0]
Oh, love it. I'm so glad to have you on the podcast because there's a lot of hype around AI at the moment. But I'm curious for what the work you're doing at Modurn. How have you leveraged AI to tackle unique challenges of large scale code based migrations and updates across enterprises? Because we often hear about this is AI, this the the game changing, the transformative technology.

[00:03:20] - [Speaker 0]
But I'd to hear more about the real world use cases and and how you're using it there. Yeah.

[00:03:25] - [Speaker 1]
Yeah. Absolutely. It's very different than I came from more research side at first when I did my graduate school at at McGill and Mila in Montreal. And to see how it's actually used in the real world was very interesting. And to really see the the nitty gritty day to day and how it can be used and leveraged in a way that you can trust it.

[00:03:45] - [Speaker 1]
So modern uses Open Rewrite, which is an open source framework that basically does the changes for you. And so these changes actually don't use ML, so machine learning. They don't use what we call AI these days. They have these it's basically code that changes code. So it's deterministic.

[00:04:03] - [Speaker 1]
It's always gonna be the same change, and that's what makes it very reliable. What's interesting, though, is you now have this framework that you can use to go and grab AI now, like, really modern AI, the ML of today, to actually do changes in a way that you can trust it. So it's kinda like Open Rewrite is this guardrail for changes or for impact analysis as well. It it lets you basically walk through code and ML only in a precise spot and precisely when it's needed, which is really good for latency, cost as well. You have basically everything that Open Rewrite can do.

[00:04:45] - [Speaker 1]
And then when it's time to have this generalist model do whatever you need it to do, you can plug it in.

[00:04:52] - [Speaker 0]
Incredibly cool. And just to boost any myths or misconceptions, what are the differences between AI techniques that might be used for individual developer assistance and those required for mass scale code refactoring? Because there's a big difference there. Right?

[00:05:08] - [Speaker 1]
Absolutely. And it it might sound a bit abstract until you have an example to tie it to. So when we talk about ML for code in general, we'll often think of ChatGPT, Copilot, all these code assistants that can generate code for you. And while those are very useful and us at Moderna, we use them every day for our own workflow, They can definitely make you go faster. Sometimes we need them to be perfect when using them at scale.

[00:05:33] - [Speaker 1]
You need to be able to trust them, and you have a human in the loop. So when it's a one to one coding within a model, you have this this feedback loop to one, okay, generate code, then the actual developer can look at the code, say, yes, no, that's not what I meant, and continue and refine. But when you're looking at scale, you kinda lose that loop and have the human in the loop. So I think one of the things where AI really shines to be for refactoring or for this actual, like, day to day task that we're trying to solve, you want to be able to use a technology called embeddings. I don't know if you've ever heard of those before.

[00:06:14] - [Speaker 1]
Basically, a numerical representation of an entity, in our case, code. And when you use these embeddings, you can now do a lot of arithmetics with them. So you can now search for things because you can now look at distance between a user query and an entity, so code. So if you're looking for something in your code or you wanna get an idea of what kind of method declarations you have in your code, it's a great tool to be able to actually visualize this. And this is done thanks to embeddings and thanks to Open Rewrite and Modern who are able to walk through the code and have this type of information.

[00:06:52] - [Speaker 1]
One of the key technology that makes Open Rewrite and Modern so strong as opposed to using code as text as input for a model or using even just the abstract syntax tree, we have something we call a lossless semantic tree, which basically is a think about an AST, but with so much more information that gets leveraged. One of the key ones is type attribution. So being able to have this extra information that would be lost otherwise, the models are able to actually go grab this extra information and that it doesn't get lost. And this is something even a human wouldn't have access to when looking in the IDE. But thanks to the byte code, we're able to have all this information for the types and really get to what does it mean for this object to be of this type.

[00:07:43] - [Speaker 1]
What does it mean when the method uses this type? And that's what the embedding is able to leverage so much more of its, context.

[00:07:52] - [Speaker 0]
And before you came on the podcast today, I was doing a little research on you. One of the things I came across was that you've written AI for mass scale code refactoring and analysis. So can you tell me a bit more about that? What inspired you to write it? And, ultimately, what the main takeaways are for software developers and organizations that may be even checking it out after listening to our conversation today?

[00:08:16] - [Speaker 1]
We really wanted to write this book for both people who are interested in using AI for mass scale code refactoring and analysis, but also for those who just in general want to integrate AI in a way that makes sense to them and not have it just be those extra little sparkles that you add on a feature. You want it to be something that's meaningful and actually bring something new. And so for both these people, we really go into details about what were exactly the problems we faced when adding AI to Open Rewrite and Moderna, and how to actually deploy these systems in a way that you can trust it, in a way that your customers can trust it, and keeping basically the power and the control into the user as opposed to the model. Because if something has a a 1% error rate, let's say, and you have a code base of a thousand switch cases, we'll say, something like that, you have a thousand instances that you wanna use your model, A 1% error rate means there's 10 cases where there could be an error. And as we know, sometimes one little change in code can make a whole difference, especially in a Swiss case.

[00:09:27] - [Speaker 1]
You can imagine how that can very disastrous effects after. So the key takeaways in in this book is we really wanna start with saying, like, a good refactoring campaign can start with understanding your code. So it kinda goes back to this embedding, going back to the LSTs who have this extra information. We also look at using the smallest model that you can use for what you need. So a a sentence we kinda coined was, don't butter your toes with a chainsaw.

[00:10:00] - [Speaker 1]
And it's really when you have these boundaries in what you're gonna deploy that you start getting very creative. And these models are so generative. They're so good at facing different kinds of problems that are not necessarily in their training set that it's a great opportunity to start stacking them a little bit like Lego blocks and to experiment. So that's another thing we really are trying to get across of try it and try with small things first. You don't go for a generative model straight from the beginning.

[00:10:32] - [Speaker 1]
Look if you can't train your own model. Look if there isn't a model that's already trained. Like, there's so so many embedding models out there that are open source, free to use, and can run on on very small, computers.

[00:10:43] - [Speaker 0]
And for any developers that are listening to our conversations, they they probably heard so many different things. Some are good, some that are bad, some that make them a little cautious. I'm curious, as someone that's right in the heart of this space, how do you see AI maybe transforming the future of software development beyond just code creation? What what are you seeing here?

[00:11:03] - [Speaker 1]
It's a really fun space to be in. As much as there's gonna be a lot of pushback, and there's sometimes gonna be people who are kinda dreaming too quick, too big. It's like, alright. This is a great tool. It's not gonna solve world hunger yet.

[00:11:17] - [Speaker 1]
But for code, I think definitely letting developers interact with their code in a different way is definitely the thing in the short run that I'm most interested in or excited to see. Even ourselves, we're working on a new way to show search results. Modern can be used to do transformations of code, whether that's refactoring, upgrading libraries, but it can also be used to search for things. So if you're looking for some sort of method or you wanna see all the methods from a certain library, that's very easy to do. When we show these results, they're gonna be showed repo by repo, file per file, and that kind of stuff.

[00:11:56] - [Speaker 1]
What if we could use AI so that you can kind of walk through your code in a different way and see it differently? If you have some results that are similar to each other within the search result, why not put them next to each other? This idea is, like, helping you get to the deep parts of your code faster even if you don't interact with that code often. So that's just one example. But this idea of able to even ask questions about your code.

[00:12:24] - [Speaker 1]
Where do this user's credentials get saved in this repo? And to be able to ask a question like this to your code without knowing what library you use to keep those credentials, that's really where we're gonna harness this day to day expertise that the code will have. And it's gonna make it a lot faster to jump into new code, I think.

[00:12:45] - [Speaker 0]
Wow. Love that. So much time that could be saved from that as well. And for business leaders listening and and all the conversations that you're having with business leaders, what would you say are some of the most common challenges that organizations are are currently facing when attempting large scale code based migrations?

[00:13:03] - [Speaker 1]
I think, that it goes back to time. Right? You were just saying about time saved. I think these migrations take so much time. Some companies will put a certain amount of time in their budget.

[00:13:14] - [Speaker 1]
Alright. We're gonna dedicate this many engineers over that much time. And it it becomes kind of a whack a mole. Right? So you have so much that you need to migrate already.

[00:13:28] - [Speaker 1]
And as you have this team working on this certain migration, whatever you're working on might already become something that needs to be migrated later. Right? It's a problem that keeps happening, and it's a snowball that we need to stop kinda early on. So having someone work on these projects at like, by hand, it's doable. It really depends on the scale and what your goals are.

[00:13:51] - [Speaker 1]
But this is kind of the problem with the origins of Open Rewrite we wanted to solve is let's do this automatically. So instead of having one person work on one repo, have one person work on a certain migration that can then be applied on all of the repos, and we all gain from that. Right? So we have a lot of migrations that are open source already, these recipes that anyone can use because it's we can all gain, basically, from having written this once, and let's just have all the changes be done at the same time instead. I think the the next goal is for these people who write frameworks and libraries.

[00:14:29] - [Speaker 1]
When they have a new version, the goal would be for them to write that migration itself as well. And it's just gonna make people be able to jump onto that new version of that framework so much faster because they can now automatically upgrade their code to whatever, version they need to use this other library or framework.

[00:14:51] - [Speaker 0]
And the amount of time that can be saved here is immense and incredibly exciting, but there will be some hardcore traditional developers and extremely cautious IT managers out there that will be a little bit cautious and a little bit more wary. So how do you put them at ease? How do you ensure that AI powered code refactoring is faster, but also maintains the reliability, the scalability, and security of enterprise solutions? Because it's something we can't ignore here as well. Right?

[00:15:21] - [Speaker 1]
I'm happy that they asked these questions. I think it's a really good process. Like I said, I come from research. Right? So this is, the day to day you have to ask yourself, what what are gonna be the hard hitting questions?

[00:15:32] - [Speaker 1]
Right? And I think here, if it's something at scale that we're using so we have even on our public platform for people to try out recipes. We have so many repos that they can see it on. They can also do it on their own code. If once they talk to our our team, we always facilitate this because we want them to be in control.

[00:15:53] - [Speaker 1]
Every change then becomes a a certain pull request or comment, whatever you wanna you wanna do. But, basically, you can verify all of these changes. But I think very quickly, you'll see that there aren't errors. And that's because of the deterministic framework, the the way that open rewrite is done. When it comes the time to make the changes, those aren't AI.

[00:16:18] - [Speaker 1]
That is a human has written, when you see this, do this. And like any code, there'll be some bugs, obviously. But the fact that we run these recipes on so many repos, we they really get this, like, stress test, basically, of quality.

[00:16:34] - [Speaker 0]
And there is in some circles concern that AI will replace jobs, etcetera, but I always maintain that the magic happens when AI and the human worker or, in this case, developers can achieve things together that neither can achieve on their own. So what role do you believe collaboration between developers and AI assistants will ultimately play in the future of software development? Because it's not about one or the other, is it? I think it is that collaboration.

[00:17:02] - [Speaker 1]
Even myself, I I saw myself using these LLMs to chat about concepts. I didn't come from the Java world. I came from Python. And to be able to add very quickly my touch and understand very quickly the framework that I was working in that was written in Java. It was so great to be able to instead of going to bug someone else who's busy doing something else, why can't I first start chatting and asking my questions in a way that makes sense?

[00:17:33] - [Speaker 1]
And it's a great first attempt in understanding code. And I'm not saying these chatbots though should replace documentation, but it is definitely another tool in your toolbox as someone who gets started either with a new framework, a new language, a new project, a new team, whatever it is. And that definitely can only help. And so I can only imagine a junior, that stress of going to go see a senior for the first few times, it could be very intimidating. And so to have this first this first line of defense in a way and that way to be really taken under the wing of AI to understand the code faster, and then you can go ask your questions.

[00:18:16] - [Speaker 1]
Right? So I think that that's very useful to have kinda like this and it again, another tool in your toolbox, and I definitely see it as cooperative.

[00:18:26] - [Speaker 0]
100% with you there, especially on that another tool in your toolbox, such a great way to look at it. And as as we look at adoption, we've been talking about AI for two years. We're now seeing a maturity. But how do you see adoption of AI for large scale code maintenance? How do you see that evolving over the next few years?

[00:18:43] - [Speaker 0]
And do you think some industries will benefit more than others or or which will benefit the most?

[00:18:48] - [Speaker 1]
I think anyone that has old repos that they're kind of afraid of touching, Some of the people who have worked on these projects are either retiring or leaving. So to have more junior people writing in COBOL, that kind of stuff, it's gonna be a good time to do these migrations and a good time to do these in a way that can make the code more easy to maintain in the future. I think it's it's gonna be interesting, the the industry. So I look at people who need to have code that is very secure and that have a lot of code, and that code tends to be old. So that's what banks, insurance companies, these they have such an amount of repos, and they're they're massive, and they're so complicated and intertwined with each other.

[00:19:37] - [Speaker 1]
And they're very hard to change as well because you there's so much security that goes behind them. I I think this is a great place to have this trustable change using the recipes.

[00:19:50] - [Speaker 0]
Completely agree with you. And at the very beginning of our conversation today, you were telling me about your incredibly cool job title, AI research engineer. But, obviously, you took a long time to get you where you are, and you've enjoyed a successful career. But I'm gonna ask you to look back for a moment now because none of us are able to achieve any degree of success without a little help along the way. Very often, it's someone that's seen something in us or invested a little time or pointed us on the right path, for example.

[00:20:17] - [Speaker 0]
But is there a particular person or people that you're grateful towards that we could maybe give a little shout out today?

[00:20:23] - [Speaker 1]
I love that question. Absolutely. I love throwing flowers. I think it's there's so many people that every day, like, I'll write a little bit of code, and it reminds me of them. Right?

[00:20:34] - [Speaker 1]
But the two that come to mind is definitely my two supervisors for my master's that I did at McGill and Mila. When I was choosing my supervisors, that matching process, an advice I got was don't get two supervisors, try to do only one. The reasoning behind that is it's hard to know who's going to be the leader of the two. And you might end up kind of falling into the cracks just because professors tend to be so busy. So I went against that advice, and I am so happy I did because I got too awesome.

[00:21:08] - [Speaker 1]
And they gave me so much time, so much of of their attention. And yeah, so Jin and Danny are just they were both incredible. And they had two very different backgrounds. Jin is more in the software engineering research, while Danny is more into the ML for code, graph neural nets. But together, the three of us, I think we really did some good work.

[00:21:30] - [Speaker 1]
And it was just I learned so much on what kind of questions to ask yourself. And like I was mentioning before, where you're always asking yourself the hard hitting questions that definitely comes from research, Everything you do, it's not great. It's working. I'm done. It's why is it working?

[00:21:48] - [Speaker 1]
And this, like, this it it gives you a really better, like, way to code and and to really push your understanding of anything you're doing so much further. And for that, I'm forever grateful.

[00:22:01] - [Speaker 0]
Oh, man. A quick shout out to Gin and Danny. So important that we share stories like this, because I suspect that both of them are unaware on the impact that they've had on your career. Equally, the people we didn't mention, that when you're writing code and you think about those individuals and what they've contributed to that code that you're writing, and all these people collectively make such a big difference in our lives. And I love the phrase you used as well about throwing flowers.

[00:22:26] - [Speaker 0]
Absolutely love it. But before I let you go, for anyone interested in learning more about Modurn or bombarding your inbox with any questions that they might have, where would you like to point everyone listening?

[00:22:39] - [Speaker 1]
I think LinkedIn is the best place. I'll I'll make sure to filter through the people asking me to use their platform and only look for the questions and messages, but definitely LinkedIn, I think, is the best way to reach me.

[00:22:50] - [Speaker 0]
Awesome. Well, I'll add a link to that and the Modurn website, and we covered so much today in a short amount of time. And so so good that you managed to bring to life everything that we're talking about with real world examples, but just a big thank you for sharing your story today.

[00:23:06] - [Speaker 1]
And thank you for having me.

[00:23:08] - [Speaker 0]
Wow. I think it's clear that scaling AI beyond individual code files to large scale enterprise applications is without doubt a transformative challenge and equally an opportunity. And as Justine highlighted today, AI driven mass scale code refactoring can empower organizations to reduce efficiencies, handle migrations more effectively, and ultimately evolve with the demands of modern development. But the conversation also raised a few important questions. How will organizations prepare for AI's growing role in enterprise software?

[00:23:43] - [Speaker 0]
What steps will they take to ensure sustainable modernization? And with AI proving its value in solving these complex challenges, we need to remember the tools we adopt today could reshape how we build tomorrow's digital infrastructure. But over to you, let me know your thoughts. Tech blog writer outlook.com. LinkedIn, x, Instagram, just at Neil C Hughes.

[00:24:04] - [Speaker 0]
Let me know. But that's it for today. So thank you, Phyllis Singh, as always. Hopefully, you can join me again tomorrow. We'll do it all again with a different guest.

[00:24:12] - [Speaker 0]
How's that sound? Good answer. Speak with you all tomorrow. Bye for now.