What does artificial intelligence really cost when an experiment becomes a production service used across an enterprise? Token consumption is easy to see, but Greg Holmes argues that it may represent only a small part of the final bill.
In this episode of Business Tech Perspectives, I speak with Greg Holmes, EMEA Field CTO at Apptio, an IBM company, about AI economics, technology business management, and the growing challenge of connecting infrastructure spending to business value. Greg works with CIOs, CFOs, and technology leaders across EMEA, where conversations that once centered on cloud cost and FinOps are increasingly focused on a simple question: is AI worth what the organization is spending?
Greg describes how teams can move surprisingly far through an AI pilot before finance sees the full consumption pattern. The token bill may then reveal a larger operating model involving data storage, data movement, cloud infrastructure, governance software, security, and the people who built and maintain the service. In examples discussed during the interview, he says direct AI costs can account for around 20 percent of the total. That figure should remain attributed because every workload and enterprise will have a different cost structure.
The measurement problem becomes harder when businesses ask for a single return-on-investment formula. Greg argues that each initiative needs a clear unit of work. A service desk system might be measured by tickets resolved, an insurance system by claims processed, and a document service by files completed accurately. Leaders can then compare the value of that completed work with its full production cost rather than treating token usage as the result.
We also discuss agentic AI, where autonomous activity can consume resources continuously and unpredictably. Greg recommends guardrails such as a spending ceiling per agent, automatic checks when consumption rises, and model tiering. A cheaper model may complete the straightforward 80 percent of tasks successfully, leaving an expensive frontier model for work that needs greater capability. This approach gives engineers room to test ideas while limiting the risk of applying the costliest system to every request.
Another concern is behavior. Greg shares examples of organizations that encouraged employees to prove they were using AI by consuming their full token allocation. Engineers responded to the metric and built tools that used tokens because appearing low on the usage chart looked bad. The organization received the behavior it rewarded, even when that behavior had little connection to a useful business result.
That lesson matters for the relationship between finance and engineering. Finance wants predictability, while technologists need room to test new ideas. Technology business management and FinOps can give both sides a shared language based on services, unit costs, and outcomes. Greg argues that visibility creates the basis for trust, allowing a team to discuss whether a service is worth funding rather than fighting over an unexplained bill.
We also examine the belief that AI will inevitably become cheaper. Models and computing methods may become more efficient, but growing demand, data center construction, energy constraints, and the creation of additional workloads could keep total spending high. Greg’s advice is to make a production service financially worthwhile from its first day rather than depending on future price reductions to rescue a weak business case.
Is your organization counting AI tokens, or can it explain the full cost and the business result produced by every AI service? Listen to the episode and share how your finance and technology teams are measuring AI value today.
Useful Links
[00:00:00] Do you need AI agents that you can trust? Well, with an AI data layer providing real-time connection within your data platforms, you can trust your agents to provide accurate solutions. So, scale your business by trusting your agentic AI accurately getting the work done for you. Trust its capabilities with Denodo. And you can do that by simply visiting denodo.com to learn more.
[00:00:27] What does an AI project really cost once it leaves the pilot and starts running across the business? The token bill might attract attention, but it can be only a fraction of the total once data storage, infrastructure, governance, security and specialist labour all enter that calculation. It's a little bit more complicated than tokenomics.
[00:00:51] So, today I'm going to be joined by Greg Holmes, EMEA Field CTO at Apptio, which is an IBM company. And together we're going to examine why enterprise AI economics become difficult at production scale. Because we're going to discuss how leaders can connect spending to a unit of business value. Why cheaper models might handle much of the work.
[00:01:17] And how badly chosen targets can persuade engineers to consume tokens rather than just improve results. So, if your AI budget is growing faster than anyone can explain it, this conversation offers a far better way to start measuring. And that is what it's all about. So, enough scene setting from me. Let me introduce you to Greg right now. So, thank you for joining me on the show today.
[00:01:47] Can you tell everyone listening a little about who you are and what you do? Thanks, Neil. So, as an avid movie fan, I spend my working week with Apptio, which is an IBM company as the field CTO. So, most of my time is really with CIOs, CFOs and tech leaders focusing around the problem of technology spend. And really, technology spend has outgrown many of the financial practices built to manage it. You know, we're 15 years in TBM and FinOps.
[00:02:16] And now we're really talking almost entirely about AI economics. And is AI really worth it? And yeah, I spend a lot of my time around London, but working across all of EMEA with leading companies around this topic. Well, there's so much I want to talk with you about today. But I've got to ask, when we talk, if we go right back to the beginning of your career and lit that spark in technology and made you want to go on a career like this, what was those sci-fi movies? Did that have any element of that?
[00:02:44] Did you even think way back then when you were watching those sci-fi movies that we would end up where we are now? Well, Neil, to be true, you know, probably dates me somewhat. But watching the early Matrix movies, you know, I kept thinking about what's this other world we could live in? And are computers really going to take over the world? And if so, do I want to be on the side of the computers and understanding where they're going and being a part of the system? Or do I want to be, you know, outside of that?
[00:03:13] So, you know, for me, it was really, really quite exciting a time to get into the technology world and follow the path of how a technology vendor supplies software to companies to help them out with their day-to-day problems. Yeah, I completely agree with you. And, of course, over the last three years, we've seen the arrival of mainstream AI. Of course, we've been talking about AI a lot longer than that. But what we've seen over the last three years has been phenomenal.
[00:03:39] And even now we're moving beyond that and AI costs are now moving towards usage-based models, arguably a little bit like cloud before it. But what are finance and technology leaders, what are they finding hardest to predict right now? Yeah. So I think a key challenge around using AI and like cloud before it is that engineers and teams have the permission almost and the right to go and use things and test them out and see how they're going.
[00:04:08] And quite often things get quite advanced before you get to a checkpoint where the bill is substantial. The volume of usage actually is surprisingly large. And that is then the moment that it starts to get properly noticed and it needs to be reviewed and checked against, you know, is this worth running? And how do we actually plan for the future use, right? How much of this new kind of capability do we really need?
[00:04:36] And do we need to think about optimizing how we use it? And is it delivering? Is it in the right place? Yeah. Again, completely agree. We've seen so many stories over the last few years around people throwing everything at AI. And then we've got the rumors around AI replacing people to some enterprises running out of tokens by Q2 and realizing that actually humans are actually cheaper to do some of those roles.
[00:05:01] So can you share an example of what you've seen where an AI project looked affordable in pilot, but then changed materially at a production scale when things quickly got more expensive? Yeah, I've certainly seen it in engineering projects in our clients.
[00:05:18] Some of the largest customers have been testing out AI in certain processes and realizing once they've rolled it out that they haven't adequately planned for how much they need in order to keep it running and to keep that new process working. And often that token bill on its own is only the smallest part, right? It's maybe 20% of the real cost of getting that initiative going.
[00:05:44] You've got to think about all the storage, all the other infrastructure around things. You've got to think about the governance software and all of the people who've been building these components too. So it's only at that production kind of moment that you start to see the operational cost rise up and you realize just how much that's going to be a struggle to get it over the line in terms of funding and make sure that it's okay to run.
[00:06:10] You know, most organizations, they'll try and experiment, they'll start doing things first. And then they come back and realize we really need a proper plan around how to do this properly. And yeah, they go and use up their tokens. I think we've seen examples where organizations have incentivized the wrong thing. So they incentivize engineers on showing that they were using AI appropriately and that they're making sure that they use up their allotted number of tokens every month.
[00:06:37] And then, you know, the number of tokens tends to be like a minimum rather than a maximum per user. And those organizations run out really quickly. You know, getting beyond just seeing AI as a cost metric, you know, you've got to see it as how is it actually adding to the value? How much is it worth funding that next, you know, next initiative on AI?
[00:07:01] Yeah, I'm glad you said that because there is a huge focus on return on investment now and measuring business value that the technology can deliver. So knowing what to measure, that can be the difficult part. So what measures or metrics can help leaders connect AI infrastructure spending to that real business result, that business outcome? Yeah, I think most organizations need to consider, are they actually including all of the costs that matter to the initiative?
[00:07:29] Is it a balanced and well understood cost? And is it operational, right? Are they looking at how does the cost change over time based on consumption by multiple different parts of the organization? So by bringing that kind of cost base together and accurately modeling that, it gives you at least the starting point. But then, as you say, what really matters? And that's the business results, right? So what are we actually trying to achieve? And then every initiative, you need to think differently.
[00:07:59] Like this initiative might be about resolving tickets quicker. That initiative might be approving customers' claims in your life for an insurance company, or it might be about producing or processing documents that you receive from clients. So being able to measure that unit of work and how much value is in that unit of work getting done and comparing that then to the operational cost and the overall production cost of getting those services up and running.
[00:08:27] So you need an end-to-end system that will attract those initiatives and be able to give some kind of business perspective on the value along with that cost perspective as well. And, of course, there has been a lot of excitement this year around agentic AI or, more importantly, agents. The problem with agents is the more you use them, the more you want, the more that you unlock ideas that you can use.
[00:08:51] And as a result, they consume resources continuously and often in unexpected ways. So what controls are you seeing companies putting in place without stopping useful experimentation and getting that balance right? Yeah. So it's really about setting up some guardrails rather than just hard gates, right? People hate seeing obstacles, but they want to be guided in the right direction.
[00:09:15] So think about things like a spend ceiling per agent with some automatic checks that might come in if it's going to go beyond that kind of automatic kind of amount. Model tiering so that you can look at the model that can handle the easy 80% of things. Maybe that's a cheaper model than using the expensive model for everything.
[00:09:39] We see a lot of engineers jumping straight into looking at frontier AI models and LLMs and looking to use those because they're so powerful. But you might also find that for some of those previous examples about processing claims or about reading documents from customers and things like that, they might be able to be achieved at the same success rate with a much cheaper model and something that's much more scalable financially for the organization.
[00:10:04] So in many cases, you need a way to look at what the right model for the right purpose is. And that comes with having a technology part of your organization that's able to advise the business on how are they designing and using AI in every single different solution that they're trying to use it.
[00:10:25] And anyone that has scrolled down their news feed this year will think that all these uncontrollable costs are completely down to AI and tokenomics, etc. But I think often they're often distracted by other costs too. So where do, let's say, cloud contracts, data movement and software licensing, these places hide costs that teams often miss too, right? Yeah, absolutely.
[00:10:49] I mean, a lot of vendors are now have their own AI embedded in their products that might have cost associated with overages and extra usage, right? So you need to be considering that as part of your AI spend as well. You also need to be thinking about all the data that you're pouring into AI, right? And if you're using AI in different deployment kind of areas, right? If you need sovereign cloud, you need to put certain data in certain locations.
[00:11:18] That usually means you're duplicating your data. So you're taking it from one area, you're storing it again in another area so that you've got safeguards around how it's used in those spaces. So you need to think about, you know, AI is not just the token spend. It's your data storage. It's your engines, pipeline engines, getting the data to the right places. It's the infrastructure and security around things. And it's hiring experts to make sure that these things are running the right way.
[00:11:46] You know, if you're looking at the overall cost, you know, maybe 20% of it's in the AI cost itself, you know, and then you're using AI to generate more workloads too, right? So some of those workloads are just traditional workloads, right? You're processing a lot of data and you're doing more with the results. And that's how organic AI is actually really powerful. And it seems that in just about every enterprise, there are more than a few conflicts there.
[00:12:12] For example, finance, they traditionally want predictability, while engineering, they want greater flexibility. So how do you see these two groups coming together and making better decisions when they're often at odds or different ends of the scale? Yeah, so this is what we've been spending many years putting into the discipline here, right? So the discipline of technology business management, the discipline around FinOps.
[00:12:37] And it's really about having a common language that connects what the engineers talk about with what the business users are talking about with the financial angle as well, right? So if you're talking about how a service is delivering value back to the business, you need a shared unit cost that you can talk about with some confidence between people about knowing what each piece is doing.
[00:13:01] So being able to have that useful information shared between those different functions and not hidden away is very important. So it's really about that visibility and the precondition for trust between those different functions so that they can both make a difference, right? The business user doesn't want a costly service. They just want it to do and achieve the business outcome. The finance team want to see, you know, maybe create metrics so that they can see the costs are improving over time.
[00:13:29] And then the technologists want to know that the new technology that they're using is worth using, right? That they want to know what's the business outcome. They do want to be connected to the business. And there will be some leaders listening that will argue that high AI costs are just temporary and optimization can wait to a little bit further on down the road. But where does that reasoning hold up and where does it actually create risk, especially when we've seen where we've come from in the past with technical debt, etc?
[00:14:00] Yeah, I would love to see a world where all of the AI is going to get much cheaper in the future. But if you think about it right now, you know, there's a lot of challenges about building new data centers. And this is political as well as technological, right? It takes energy to run a data center. And that energy needs to come from somewhere. And we want it to be sustainable as a society, right? We don't want to be, you know, destroying the planet just to run more workloads in the cloud or on AI.
[00:14:28] So really what we need to see is that we are making more efficient ways to run AI, that we're creating better models and not just better from a size point of view, but also from an efficiency kind of point of view. And then we're also looking at how we're getting business value out of these AI services, right? If the cost doesn't come down because more people want it, we'll end up with demand pushing the prices up rather than down.
[00:14:55] So we shouldn't just sit back and wait and hope that the costs will save us in the long run. I think it's great to have a little bit of experimentation. But once you're sizing things up for production, you want to make sure that it is worth running on day one and that it's going to work for your organization. And I'm curious, how are you seeing cost pressure affect the people that are building and then using AI when the targets keep changing faster than systems can be redesigned?
[00:15:24] Because the pace of technological change is phenomenal now, but there's also that realization it might never move this slow again. It just keeps getting more and more. But what are you seeing now? Yeah, I think, yeah, you've got to be careful about the kinds of metrics you run teams on, right? As soon as you identify a target metric for a team, they will start optimizing their behavior around the metric itself and regardless of the outcome, right? So this is how the whole token maxing kind of problem emerged.
[00:15:53] Engineers were actually designing tools to just use up their tokens so that they could look good on the token charts, right? That they weren't at the bottom saying using the least AI to do their jobs. So people want you want to remove the fear from people from doing the right thing just because the bill looks bad or that finance is worried about where things are going. You want to give teams, say, a budget that they can be set as a target for themselves rather than be judged against.
[00:16:22] And you want to give that ownership to the teams so that their behavior is around producing the result that the business really wants. And at Aptio, enterprises come to you for AI-powered data insights and help them make smarter financial and operational decisions. And again, I'm curious, if we put all those conversations you're having with customers and clients and inquiries and conversations on the show floor at tech conferences,
[00:16:49] are there any trends in what's concerning people now, what they're talking about, and how are you helping them overcome that? Yeah, I think so. I think some of the trends we've already talked about today, but the whole sustainability of IT and AI is a big area. We see some of our customers actually doing a great job in their organizations and getting recognition for the work they're doing.
[00:17:14] I'd call out NatWest, who are a winner of a TBM award last year for their work at sustainability and showing the energy used by their workloads is actually worth it from their organizational point of view. What I love seeing is some of our customers getting case studies out there to talk about those business results that they're achieving, whether it's cost savings, whether it's actual rationalizing their application portfolio or getting the costing of their AI right.
[00:17:41] All of those things really get out there and show people what we do is really worthwhile. So, yeah, highly recommend looking those up. You know, it's easy to find some of these things on my LinkedIn profile. You know, so Greg Holmes at Appdio, you know, I post for most weeks about this. I'd highly suggest people look at things like the Technology Business Management Conference this year in Miami. It's going to be a great way to see leading practitioners talk about their achievements.
[00:18:12] And when I say leading practitioners, they're not IBM people. They're the customers, right? They're the people doing these implementations, earning the results, showcasing their numbers, right? So these people really believe that the results that they are doing is super important for their businesses. And it gives the business then the ability to confidently invest into the right areas for their organization and choose the priorities accordingly.
[00:18:38] I love events like that that are showcasing what the customers are doing more than anything else. And when is that? When's that coming up? These other days? Early in November. It's the second week in November in sunny Miami. We'll be on the beach, Neil. Good chance to enjoy some sunshine later in the year. You know, we get some of the leading clients from Europe to join us at that event, plus a lot of people who are curious and wanting to get into the discipline as well.
[00:19:07] So you don't have to be a technology user at that point. You just have to be curious and interested about how could this help your organization and how can you get better results in this area? Yeah. Brilliant. Well, I will have links to everything you mentioned, along with a few other things that I can find that I think will help people listening. So go over to techtalksnetwork.com. There will be a blog post associated with this episode. We'll put all the useful links together. And hopefully we can meet at some point as well.
[00:19:35] I love to hear what people listening think about this. But, Martin, thanks for your time today. Really appreciate it. Thanks very much, Neil. I think one of Greg's big messages today is that token usage only tells a small part of the AI cost story. Leaders need to understand every production expense, agree on the unit of work that the system performs, and compare the cost with the results that the business receives.
[00:20:03] And this is a massive lesson, I think. It gives teams budgets and guardrails rather than targets that reward consumption. And model tiering can also reserve expensive frontier systems for the smaller group of tasks that genuinely require them. So waiting for AI to become cheaper, yeah, that might sound reasonable, but demand, energy requirements, data centers, and new workloads.
[00:20:29] I think it's fair to say that all these things will collectively keep the pressure on prices. So a big thank you to Greg and Aptio for joining me. Over to you. Is your organization measuring AI by tokens consumed, money saved, work completed, or a business result that finance and engineering can both trust? I'd love to find out what you're going through at the moment. So techtalksnetwork.com. You can leave me an audio message there.
[00:20:58] Connect with me on LinkedIn at Neil C. Hughes. Lots of different ways you can find me. But I'd love to hear from you. But that is it for today. I'm out of touch. You're out of time. So I will return again very soon with another guest now. Thanks for listening. Bye for now.