What prevents a successful AI experiment from becoming a dependable production system that delivers measurable business value?
In this episode of Tech Talks Daily, I speak with Ed Macosky, Chief Product and Technology Officer at Boomi, about AI pilot purgatory, integration, governance, model selection, token costs, and the technical skills businesses may need as adoption grows.
Ed leads Boomi's product and engineering teams while also using AI tools inside his own organization. That gives him a view from both sides: creating technology for enterprise customers and applying it within active product development workflows.
He believes many AI pilots begin with the wrong question. Teams become interested in the latest model or feature before defining the business problem they want to solve. The experiment may work during a demonstration, then fail when it encounters real data, access controls, security policies, and production systems.
Placing company information inside a data lake and adding a language model does not automatically create a business application. The system must access current data reliably, respect employee permissions, connect with existing applications, and operate within governance rules that security teams can approve.
Ed recommends beginning with a defined business opportunity and establishing the access required to support it. Existing APIs can already provide authentication, permissions, and governance. MCP can offer another route into enterprise systems, but those connections still require security, monitoring, and management.
Team alignment also matters. An AI center may be racing to test models while an integration center concentrates on a different set of priorities. When those groups fail to coordinate, the pilot lacks the connectivity and automation required to become part of a production workflow.
The discussion then turns toward fragmentation. Every technology wave produces new vendors, frameworks, and specialist tools. Early experimentation benefits from variety, but mature companies can eventually find themselves maintaining a complicated collection of products held together with custom code and, occasionally, the digital equivalent of duct tape.
Ed does not recommend placing every function with one provider. He does argue for enough consolidation and abstraction to prevent experimentation from creating years of technology debt. Governance and observability should also work horizontally across different environments, including platforms such as SAP, Salesforce, and several AI model providers.
That becomes increasingly important as businesses introduce autonomous agents. Leaders need to know which agents exist, what systems they can access, what actions they can take, and how each decision is recorded. AI gateways and agent control towers can provide a wider view across otherwise separate technology environments.
Ed also introduces the idea of the frontier engineer. A prompt engineer concentrates on communicating effectively with a model. A frontier engineer understands how the model works, including its logic, mathematics, algorithms, and suitability for different workloads.
He does not believe every company needs a large team of these specialists. However, he argues that enterprises need at least one person capable of assessing vendor claims and deciding whether a frontier model, specialist model, or open weight model fits a particular workload.
Cost creates another reason to examine model selection. Sending every employee request or agent task to the most capable frontier model can become expensive. Some repeatable workloads may run on open weight models inside the company's cloud or hardware environment, giving finance teams greater cost certainty.
Boomi is developing Boomi Prompt to route requests according to their complexity and requirements. A simple factual request might go directly to an API. A forecasting task may use a smaller model. A difficult analytical request could be sent to a frontier model.
Ed uses the weather as a helpful example. Retrieving next Tuesday's forecast does not require a language model when a public weather API can return the answer directly. Asking a model to perform every form of automation wastes tokens, computing power, energy, and money.
The episode closes with practical advice for CIOs. Avoid starting with a broad objective such as agentifying the entire business. Choose a department, identify a small number of tasks, define the expected return, and work backward. Once the team proves value and understands the operating requirements, it can repeat the process elsewhere.
Could intelligent routing, stronger integration, and clearer business outcomes finally move enterprise AI beyond pilot purgatory? Listen to the episode and share your thoughts with me.
Useful Links
Connect with Ed Macosky
Learn More About Boomi
[00:00:00] The leading issue of agentic AI in businesses right now is ensuring agents act with compliance guidelines. And Denodo applies guardrails across your entire data estate. By aligning your company's data infrastructure under one system, these guardrails perform consistently across your platform. So start scaling your business and start with Denodo. Simply visit Denodo.com to learn more.
[00:00:27] What if the reason that so many enterprise AI projects never escape pilot mode? Maybe it's got nothing to do with the AI model itself. Well, businesses right now are racing to experiment with agents and frontier models. But when those projects meet the reality of fragmented data, security controls, APIs, governance and legacy systems,
[00:00:56] the path to production can quickly become much harder. Well, my guest today is Chief Product and Technology Officer at Boomi. And we're going to discuss what it takes to move AI from experiments into everyday business workflows. And we will also talk about the rise of the AI frontier engineer, why enterprises need to understand the tech beneath the prompt, and the rapidly growing economics of AI.
[00:01:24] Because a thousand of employees and agents start consuming tokens at a rapid rate, businesses are already beginning to ask a surprisingly simple question. Does every task really need AI at all? We've certainly seen some changing attitudes there. But enough from me. Let me introduce you to today's guest right now. So thank you for joining me on the show today. And Kenny Teivon listening a little about who you are and what you do.
[00:01:54] Yeah, Neil, thanks for having me. I am the Chief Product and Technology Officer here at Boomi. So in the context of AI, not only do I run a product and engineering team that delivers products to help with AI activation and that sort of thing, but I'm also a practitioner myself. I run product and engineering teams that leverage AI tooling in order to become more and more productive. Well, thank you for taking the time to sit down with me today.
[00:02:20] And there's so much I want to talk with you about because I think enterprises have spent the last few years running AI pilots. Many are still struggling to move them into production. The thing called pilot purgatory is a phrase I hear a lot. But what is it that breaks down a successful experiment and an AI system that genuinely delivers value across the business? What do you see here? Well, there's a lot that breaks down that I see in a landscape.
[00:02:45] But most times what I find or what we find is that companies and technologists focus too much on the technology. They're too enamored with the latest, greatest models that are out there, the latest, greatest features that Frontier or other model providers are providing. And they take on projects that are technology-centric versus business outcome-centric.
[00:03:10] And they don't take into account what the business outcome should be and or the data and systems that need to support the outcome. And how they then properly access in a reliable, secure way the backend data. And I don't just mean data lakes because that's what a lot of folks do. They'll put data in a data lake. They use a large language model or something on top of that and say, great, I've got my AI project. And then they go to put it in production. It's hallucinating.
[00:03:38] It doesn't under, you know, can access systems securely. Security jumps in, puts up walls. You know, things are unsecure. They didn't think about role-based access, et cetera. But most times it comes down to accurately accessing high-quality data. Yeah. And it's so reassuring to hear you say that.
[00:04:00] And I think just to reiterate there that scaling AI takes so much more than just choosing the right model or getting blindsided by the shiny technology. So if we would just zoom out for a moment, what are the foundations that companies need across data integration, automation, APIs, and governance before AI can naturally become a part of a real business workflow? It's those foundations I think we don't talk about enough. Agreed.
[00:04:55] Agreed.
[00:05:26] So if we're going to talk about a lot of tools in an AI project, MCP obviously is very popular, but you need to secure and govern and manage that. Traditional database access with MCP, et cetera. A strong foundation layer to connect and govern and manage your enterprise systems really needs focus. And you need to make sure that your house is in order at that layer before really pushing your AI project into production.
[00:05:55] So going back to, I mean, even the previous question, just that needs to be accounted for in the project. Many organizations have these and some of the things that I'm seeing are AI teams are often running trying to solve an issue or solve this challenge or opportunity. And integration centers of excellence are often running on different projects and priorities, really aligning those priorities to team up and take stacks, connectivity, integration, automation stacks that are already in an enterprise.
[00:06:24] Make sure they're rock solid for an AI project and often running too many times. They're not partnering well enough to really bring these things to life. And I suspect we'll also have people listening inside enterprises that have accumulated disconnected AI tools, agents, models, and indeed platforms. So for what you're seeing here, what kind of problems does this kind of fragmentation create further down the line?
[00:06:52] And why is unification becoming so important for visibility, control, business performance, and so many other things? What are you seeing here? Yeah, it's traditional technology cycles, right? As new technologies emerge, startup, you know, it's like a gold rush. Everybody's running for the gold and all these startups and fragmented tools come about and all these frameworks are developed. And the issue, it's starting to rear its head.
[00:07:20] You know, at first, when the technology wave comes in, people get very enamored with all of these best-in-breed technologies and assemble these complicated, fancy stacks. But it's coming to a head in many enterprises today where it's, who's going to maintain all this stuff? And, you know, do I need 15 people with 15 different skill sets understanding these 15 different parts?
[00:07:46] And, you know, focusing on – you know, I wouldn't suggest vendor lock-in and focusing on one vendor for all of these by any means. But focusing on, you know, finding tools that properly handle and abstract a number of these technology choices to save you from the heartburn that will come along with tech debt in the future. I think about tech debt a lot. It's just an engineering leader. How do I keep things in order? How do I manage them long-term?
[00:08:16] How do I have a proper infrastructure? And some level of consolidation versus, you know, picking all of the shiny objects and trying to duct tape them all together, I would certainly advise. Particularly for mature enterprises. Yes, you can be on the bleeding edge. You can test these things out to learn and understand.
[00:08:37] But when moving things into production, make sure you're consolidating, minimizing, working with the best, stronger enterprise-grade tools that you can. Well, in the first six months of this year, I attended 16 different tech conferences all around the world. And predictably, every single one of them was talking about agentic AI and unleashing thousands of AI agents across an organization.
[00:09:04] And it got me thinking, as companies deploy more and more AI agents, it becomes the norm. How do they maintain control over what those agents can access, what actions they can take, and also how those decisions are monitored and audited without slowing down adoption? Again, the security aspect of it is something we don't talk about enough sometimes. Yeah, there are tools out there. Boomi offers them as well.
[00:09:30] But you really need to focus on your observability and governance layers and make sure those are in place for AI. And particularly for observability and governance layers, I would encourage the audience here to think about who can help you with observability and governance across your entire landscape. A lot of these tools are emerging, and a lot of companies, as you mentioned, there's all these conferences, and there's a lot of folks out there talking about these layers as well.
[00:09:59] But an enterprise has multiple software ecosystems within their business. It's pretty common for an enterprise to have an SAP environment and also have Salesforce and other ecosystems.
[00:10:13] But when it comes to running agent, and now you have frontier model technologies and others that are bringing agentic technologies into your business, getting your arms around all of that from an AI and an agentic perspective really requires you to think about horizontal layers of technology around governance and observability.
[00:10:34] So, you know, think AI gateways, AI control towers or agent control towers that span across all of your ecosystems to give you the single pane of glass. So you're not swivel chairing yourself trying to understand what are the different agents from the different ecosystems and what are they doing? And before you join me on the podcast today, I was doing a little research on you, and I was reading how you've also highlighted the emergence of the frontier engineer.
[00:11:01] Sounds great. It even rhymes. But you state that as an increasingly important role. But what does a frontier engineer understand that maybe a prompt engineer doesn't? And why do you think most enterprises will need that capability in-house in the months and years ahead? Yeah, the thing I and we, you know, I'll speak for myself and not everyone at Boomi, but the thing I think about a lot here is, you know, there's the, you know,
[00:11:32] a lot of folks are following some of these AI and frontier model providers and following them towards, okay, leverage our tools, leverage our technology. And prompt engineering is a consumer, a prompt engineer, super valuable because understanding how prompts, you know, are translated and understood, etc. within the models is very important. How to optimize those is very important.
[00:11:55] But over time, if you as an organization don't understand or have humans that understand how the models work, like a frontier engineer that's understanding how these models, these frontier models are working, understanding the logic, even some of the math and the algorithms and things behind it, you don't need, you don't need droves of these people. But if you don't understand that, you won't be able to be educated and understand the decisions
[00:12:19] you need to make around what are the best models and what models should I use within my business for my workloads. Not every model and every model provider is great for every use case out there. Some model providers are really good at software engineering and they even talk about our models are primarily tuned for helping the software developer. Well, there are other models that are emerging around, okay, industry-specific models and the different open-weight models and so on and so forth.
[00:12:49] If you don't have at least one frontier engineer that understands how some of these work, you're going on trust that you're saying what the vendor tells you is true. Now, okay, it could work for some, but in this world of AI where there's a lot of embellishment, there's a lot of overhype, etc., I just really think it's in your best interest to get at least one person
[00:13:15] who has the understanding, can get in-depth and help you with making these decisions around which models, whether it be a frontier model or an open-weight model, for your workload and for your projects as appropriate, versus going all in on one particular model vendor. Another big topic right now is the rising AI costs. They can grow quickly when employees, applications, and agents are all automatically sending requests to expensive frontier models there.
[00:13:44] So how should companies start thinking about the big buzzword at the moment, token economics, and choose the right model for each task? Smaller companies, etc., ones that aren't doing heavy volumes and load, frontier models are fine. You know, in small-scale, or they're great, actually, in many of these smaller-scale use cases. For enterprises that have hundreds, thousands of employees that are leveraging,
[00:14:12] that they want leveraging AI to continue driving their business, and particularly when you unleash AI where you don't have the end user as a proper prompt engineer, if you have finance people using AI-based tooling, whether it be Cowork or GPT or what have you, without proper prompt engineering, you could spend a lot of frivolous money through these frontier model providers.
[00:14:41] You can have someone from a department generating cat pictures all day. You don't know, right? I mean, there's some tooling that can govern that. But you're spending money on that. And as the model providers, these frontiers, as the models get better, the next model gets more expensive. And yeah, the previous models get cheaper. So there's this game where they're trying to capture all of these prompts and token usage to drive the value of these companies.
[00:15:08] But I encourage enterprise, take control back. You don't need to rely on these frontier model providers for everything within your organization. They will blaze the trail for us all, and they are making amazing models, and they keep pushing the boundaries forward. But just behind them are these open-weight models, which open-weight, for those who may not be familiar with the term, very similar to open-source. They're just not open-source models.
[00:15:37] Open-weight is, if you think of models that are open-source, that you can use to run in your own environments. And you can use your own CPUs, GPUs, et cetera, to run these. And you essentially can do it without the middleman charging you for tokens. And so if you shift a number of these, I'll call them, none of them will be predictable. That's the whole point of AI.
[00:16:05] But when you can run predictable, agentic, or AI-based workloads, and you can pair those workloads with models that are fit for purpose for the job, you can run them yourself without paying the frontier model providers. You can run them in your own cloud-based environments or your own hardware if you choose to do so. And that helps you get out of the tokenomics race of, oh my gosh, what is the frontier provider going to charge me next week, to, hey, I know this is a fixed cost.
[00:16:34] This workload is running on this, and it just will put your CFO at ease. Now, yes, there are routing techniques and those types of things where you can then have that just in time as workloads are running or agents are doing work for you. They can make decisions to upgrade and leverage frontier models if they can't execute in an open-weight model.
[00:16:57] And that's where you can experiment with that, and you can get very creative on how you do that. But at the end of the day, you don't have to, you know, I can make one point. You don't have to just go all in on Anthropic and just pay them and submit as an enterprise and say, hey, I'm at their mercy. If they want to charge me double for Fable next week, oh my gosh, now my cost has doubled. You don't have to do that.
[00:17:24] You can use older models and or you can use open-weight models to fix your cost. And as for Boomi, I was also reading Boomi's developing Boomi prompt to intelligently route requests based on factors such as cost and suitability rather than defaulting to the most powerful model. Sounds such a simple thing, but it's such a cost-effective thing and so important. So to bring that to life, how could or how do you see intelligent model routing changing the economics of enterprise AI adoption?
[00:17:53] Because it feels like a much-needed massive change. Yeah, exactly. You know, kind of playing off what I was just saying there. Yes, you should. We are introducing technology to help with intelligent prompt and workload model routing, etc. to route the requests that are coming into AI from the humans or other AI agents, etc. to the appropriate destination to run the appropriate workload.
[00:18:22] Sounds like such a novel basic idea. But hey, as I was just talking, not every ask of an agent or a model needs to go to the most expensive model. But sometimes it does. Maybe a lower model is not the most appropriate. So yes, route it. Prompt routing is okay. Route that request to go run within this model. It gets out in front of, in many cases, the prompts. Go run it here against this model.
[00:18:49] Or, hey, this is someone just asking me, what's the weather in California today? Okay. That decision may not even be going to a model. And this is something we're working on as well. You can go call a public API that just says, what's the weather in California today? You don't need to burn a token or leverage AI to do deterministic workloads.
[00:19:13] So if it's, hey, figure out what you'll predict the weather will be in California for the next week, maybe go to a basic model. You don't have to go to a very, very powerful model like Fable for that. So prompt routing can do that. So predict the weather for the next week. Go to a basic model. Oh, I want to know what the weather is predicted for next or what the forecast is for next Tuesday. Say, don't ask a model. Go directly to a public weather API service and say, get me the weather for Tuesday.
[00:19:42] I don't even have to burn AI for that. There's some, where we sit and where I've been working, I've been integrating systems and running deterministic integrations, automations, et cetera, for two decades now. And what kind of drives me nuts is, you know, AI is the hammer and everything's the nail to use that old adage, right? So everything people are trying to automate today, they're leveraging AI to do so and they're burning tokens or they're burning GPU power, et cetera.
[00:20:09] And it's, these things already exist in the forms of just basic APIs today. Just call a deterministic API, go get what you need back and move on. So we, we are helping enterprises with routing across all of these. It may be just a traditional workflow. It may be an API call. It may be a lower open weight model, or you may need the powerful model based on the workload.
[00:20:30] We're helping with that routing so that we can save money, save the world from just burning all this GPU and CPU power and this energy and so on and so forth. If we can help inject ourselves there and just make the world a little bit better and give us a little more longevity here, that we've done a good thing. Really have. And it's so refreshing to hear you say that. And if we do have any CIOs listening anywhere in the world today, maybe they're nodding at their head in agreement with what you've said.
[00:20:58] And especially if they've got dozens of AI experiments underway, but very little running reliably in production right now. Any advice you'd give or practical steps that they should maybe take to move from those disconnected pilots to governed enterprise wide AI adoption that ultimately delivers measurable business value? That's the big focus right now. Any advice on how to get there? Yes. Yes. Get very specific in the business value and work backwards.
[00:21:30] All too often I see, I hear CIOs, even I have conversations. I want to agentify my business. Okay. That's way too broad. And how do you, how are you going to measure that? Well, I don't know. I want to automate everything. I want to see ROI, et cetera. Okay. Pick a department, pick a team, pick a set of humans and focus on their tasks and then understand the ROI you want to measure and work backwards. Too many times it's I'm automating. Now what's my ROI?
[00:21:59] What's ROI? Where do you want to save? Where do you think you can save? Where do you think you can agentify something and work backwards? And what goes along with that is, so start small, identify one to five, something from one to three, prove value there, really understand it, and then rinse and repeat. And before you know it, you'll be at scale. If you try to start at scale, that is another area where too many, it's too vague, it's too big picture.
[00:22:27] Get very, very, very pragmatic and practical and identify your ROI metrics first and work backwards. Awesome. And I think that's a powerful moment to end on. And if there is anybody listening wants to find out more information on that path from pilots to scale, learn more about the importance of unification, et cetera, where can they find out more information? And in particular, the new tool you mentioned now at Boomi as well. Where can everyone find out more information?
[00:22:57] Very simple. Just visit us at boomi.com, B-O-O-M-I.com. Awesome. Well, I will add links to that, including your LinkedIn as well, for anyone that would like to connect with you and learn more about the AI frontier engineer and anything else we talked about today and the tokenomics, cutting costs, et cetera. We covered a lot in a 25-minute podcast today, but thank you so much for bringing all this to life and delivering real value yourself. Thanks again. Thank you.
[00:23:25] I think one of my biggest takeaways from the conversation today is Ed's warning against treating AI as a hammer when every business problem starts looking like a nail. Yep, AI for that. AI for this. Because sometimes you need a powerful frontier model. Sometimes a smaller model will do the job perfectly well. And sometimes, as Ed reminded us, a simple API call can deliver exactly what you need sometimes without burning a single AI token.
[00:23:55] And I think this becomes increasingly important as enterprises move from a handful of experiments to hundreds or thousands of AI-powered interactions all running simultaneously across a business. So choosing the right technology for each workload could become as important as choosing the right AI model, both for controlling costs and building systems that can operate reliably at scale.
[00:24:22] But maybe the best advice for every CIO came at the end there. Don't begin trying to agentify your entire business. Pick a specific team. Identify the business outcome and ROI you want to achieve. And then work backwards from there. Prove the value. Learn from it and repeat. Love that. So many big takeaways. But over to you. I'd love to hear your thoughts. As your AI spending grows, are you and your business paying enough attention to what sits underneath your pilots?
[00:24:52] And what will it actually cost to run them at scale? Same as always, techtalksnetwork.com. You can leave me an audio message there. Learn how you can meet me on the road. I'm attending a lot of events from September to Christmas. So let me know. And we can grab a hot coffee or a cold beer. But that's it for now. So I'll be back again tomorrow with another guest. But thanks for listening as always. And I'll see you again tomorrow. Bye for now. Bye. Bye.
[00:25:26] Bye.