What prevents a successful AI experiment from becoming a dependable production system that delivers measurable business value?
In this episode of Tech Talks Daily, I speak with Ed Macosky, Chief Product and Technology Officer at Boomi, about AI pilot purgatory, integration, governance, model selection, token costs, and the technical skills businesses may need as adoption grows.
Ed leads Boomi's product and engineering teams while also using AI tools inside his own organization. That gives him a view from both sides: creating technology for enterprise customers and applying it within active product development workflows.
He believes many AI pilots begin with the wrong question. Teams become interested in the latest model or feature before defining the business problem they want to solve. The experiment may work during a demonstration, then fail when it encounters real data, access controls, security policies, and production systems.
Placing company information inside a data lake and adding a language model does not automatically create a business application. The system must access current data reliably, respect employee permissions, connect with existing applications, and operate within governance rules that security teams can approve.
Ed recommends beginning with a defined business opportunity and establishing the access required to support it. Existing APIs can already provide authentication, permissions, and governance. MCP can offer another route into enterprise systems, but those connections still require security, monitoring, and management.
Team alignment also matters. An AI center may be racing to test models while an integration center concentrates on a different set of priorities. When those groups fail to coordinate, the pilot lacks the connectivity and automation required to become part of a production workflow.
The discussion then turns toward fragmentation. Every technology wave produces new vendors, frameworks, and specialist tools. Early experimentation benefits from variety, but mature companies can eventually find themselves maintaining a complicated collection of products held together with custom code and, occasionally, the digital equivalent of duct tape.
Ed does not recommend placing every function with one provider. He does argue for enough consolidation and abstraction to prevent experimentation from creating years of technology debt. Governance and observability should also work horizontally across different environments, including platforms such as SAP, Salesforce, and several AI model providers.
That becomes increasingly important as businesses introduce autonomous agents. Leaders need to know which agents exist, what systems they can access, what actions they can take, and how each decision is recorded. AI gateways and agent control towers can provide a wider view across otherwise separate technology environments.
Ed also introduces the idea of the frontier engineer. A prompt engineer concentrates on communicating effectively with a model. A frontier engineer understands how the model works, including its logic, mathematics, algorithms, and suitability for different workloads.
He does not believe every company needs a large team of these specialists. However, he argues that enterprises need at least one person capable of assessing vendor claims and deciding whether a frontier model, specialist model, or open weight model fits a particular workload.
Cost creates another reason to examine model selection. Sending every employee request or agent task to the most capable frontier model can become expensive. Some repeatable workloads may run on open weight models inside the company's cloud or hardware environment, giving finance teams greater cost certainty.
Boomi is developing Boomi Prompt to route requests according to their complexity and requirements. A simple factual request might go directly to an API. A forecasting task may use a smaller model. A difficult analytical request could be sent to a frontier model.
Ed uses the weather as a helpful example. Retrieving next Tuesday's forecast does not require a language model when a public weather API can return the answer directly. Asking a model to perform every form of automation wastes tokens, computing power, energy, and money.
The episode closes with practical advice for CIOs. Avoid starting with a broad objective such as agentifying the entire business. Choose a department, identify a small number of tasks, define the expected return, and work backward. Once the team proves value and understands the operating requirements, it can repeat the process elsewhere.
Could intelligent routing, stronger integration, and clearer business outcomes finally move enterprise AI beyond pilot purgatory? Listen to the episode and share your thoughts with me.