Measuring Whether AI Is Actually Working With Nexthink
AI at WorkSeptember 12, 2026
52
00:23:5721.93 MB

Measuring Whether AI Is Actually Working With Nexthink

How can a business tell whether workplace AI is producing a meaningful result rather than another encouraging adoption chart?

In this episode of AI at Work, I speak with Scott Pope, Director of Value Advisory at Nexthink, about a problem facing many technology leaders. AI tools are reaching employees quickly, but deployment, usage, and business value are often treated as though they describe the same thing. They do not. A company can distribute thousands of licenses and report active users without knowing whether work became faster, easier, less expensive, or less frustrating.

Scott argues that AI value has to be defined before a rollout begins. Productivity may matter most to a chief executive or HR leader, while a CFO may focus on cost and an IT support manager may watch ticket volumes. Each stakeholder is working with a different currency of value. Without a baseline, the business cannot measure the gap between its starting point and the result it hopes to achieve.

That distinction matters because familiar IT measurements can create a misleading picture. Scott says a decline in support tickets does not automatically prove that the employee experience improved. People may have stopped reporting problems, created workarounds, or accepted friction as part of the job. Infrastructure can appear healthy while employees continue to lose time at the device, application, or workflow level.

We discuss why digital employee experience, often shortened to DEX, has moved from a specialist IT concern into a wider business conversation. Work happens where employees interact with laptops, virtual desktops, mobile devices, applications, and services. Monitoring servers and cloud platforms remains useful, but it does not reveal every delay, failed interaction, or workaround experienced by the person trying to complete a task.

Scott explains how observability can help organizations understand which AI tools employees are using, where adoption is deep or shallow, and which teams may need support. He is also careful to distinguish visibility from proof of value. Knowing that an employee opened an AI application is a starting point. It does not show whether the tool saved time, improved a decision, reduced cost, or produced a better customer result.

The conversation also considers why one AI tool will not suit every role. Different teams work with different information, processes, risks, and desired outcomes. A persona based approach can help a business decide which technology fits the work rather than asking every employee to adopt the same product. It can also reveal where people need timely guidance instead of a training session delivered once and quickly forgotten.

For Scott, the people question is where many programs become difficult. Providing access to software has become relatively straightforward, but changing established behavior takes communication, evidence, and a reason employees can recognize in their own work. Leaders often explain what AI could do for the business while giving less attention to the personal value for the person expected to use it.

The opportunity is a workplace where technology problems are identified earlier, employees receive help at the moment they need it, and AI investments can be connected with measurable results. The risk is that businesses mistake purchasing and activity for progress while adoption becomes uneven and employees quietly carry the cost of poor implementation.

Does your organization know what its AI tools are changing for employees, and which measure would give you the clearest answer? Listen to the episode and share your thoughts.

Useful Links

[00:00:00] Do you need AI agents that you can trust? Well, with an AI data layer providing real-time connection within your data platforms, you can trust your agents to provide accurate solutions. So, scale your business by trusting your agentic AI accurately getting the work done for you. Trust its capabilities with Denodo. And you can do that by simply visiting denodo.com to learn more.

[00:00:31] What does AI success look like when license counts, usage dashboards and falling ticket volumes all give leaders maybe the wrong impression? Well, today on AI at Work, I'm going to be joined by Scott Pope, Director of Value Advisory at Nexthink. And together, we're going to discuss how companies can measure whether AI is genuinely improving work.

[00:00:55] And Scott will argue today that value has to be defined before deployment, has to be baselined against today's performance and then translated into a currency that each unique stakeholder understands. Whether that means productivity, cost, employee time or reduced friction. Value will mean so many different things to different people. And we'll also examine why one AI tool will never suit every employee.

[00:01:23] And understand what digital employee experience data can reveal about data and why technology actually remains the easiest part of workplace change. So, today's conversation may make you question whether fewer support tickets deserve the celebratory dashboard after all. We'll deliver a few surprises along the way, I'm sure. But enough for me. Let me introduce you to Scott right now.

[00:01:54] So, thank you for joining me on the podcast today. Can you tell everyone listening a little about who you are and what you do? Yeah. So, thanks for having me on, Neil, to meet you. My name's Scott Pope. I'm the Director of Value Advisory at Nexthink. For those that don't know Nexthink, we are the market-leading digital employee experience platform. And my role is all about making sure that customers see value in our product.

[00:02:23] And then we deliver that value to the customer during the implementation and post-sale piece and the customer's journey and lifecycle. My background is actually on the customer side. So, I've spent 20 years in enterprise IT. And then I moved into a role with Nexthink around four years ago. Well, thank you for sitting down with me today.

[00:02:52] There's a lot I want to talk with you about because there's been so much noise around AI. In particular, about 18 months ago, there was a lot of focus on the lack of ROI from AI projects. And I was reading how you recently said that AI has changed the game because SaaS vendors and customers are increasingly trading in the exact same currency, which is now outcomes and measurable business value. And it's so refreshing to get back to that.

[00:03:18] So, what does that change about how enterprise technology needs to be built, sold, brought, and measured, of course, too? Yeah, I mean, from my perspective and from our perspective, I think we do see a value gap with AI. But I think with value, value can mean so many different things to so many different people. And it doesn't exist until you've defined it. So, you have to define it before you start.

[00:03:47] And I think what we're seeing is that a lot of companies are just pushing out AI tools to the business and seeing what happens. So, I really think you have to focus on the outcome and then understand the currency in terms of how you want to measure value. Whether that currency is productivity. If you're working with someone in HR or a CEO, that may be productivity. In other areas of a CFO, that's probably going to be cost. So, it really depends on the stakeholder.

[00:04:19] And you really have to understand where you are today. So, you have to baseline. If you don't know where you are, you cannot see the improvement. And the improvement is the value. That gap in between is the value that you're providing to the business. I just wanted to bring some stats in here as well. I mean, the latest ONS data shows UK businesses and AI adoption has predictably risen from 12% to 35%.

[00:04:47] But the average adopter is using only 1.6 AI technologies. And just 10% report extensive use. So, what does that tell us about the difference between experimenting with AI and actually changing how a business operates and getting that value? Yeah, I think we sort of see similar from some of our own data as well.

[00:05:11] You know, the most used tool in enterprise obviously is Microsoft Copilot. Yeah. Like, we can all guess that, right? But I think one of the big challenges there that we see across organizations is just general visibility or observability across the estate in terms of what AI tools are actually being used from an independent view.

[00:05:41] You can't, if you can't see it, you don't know what people are using. Exactly. Exactly. So, I also feel, and we also see from the data, that there isn't a one-size-fits-all to AI adoption. So, we see a lot of customers leaning heavily on Copilot, but that's not necessarily the right tool for the right persona in the business.

[00:06:09] So, you have to have data to be able to see what, if people aren't using your enterprise-led tools, you need to understand what tools they are using to get their job done. And I think the key element there is having that foundational level of data and observability. And that's where, obviously, a DEX platform can help.

[00:06:37] And I'm glad you mentioned DEX platform there, because before you joined me today, I was reading how, this year, you was at the Gartner Digital Workplace Conference, where you watched DEX move from a relatively niche IT topic to center stage. So, why has digital employee experience become a board-level business issue now? And what are leaders beginning to understand that maybe they probably missed a few years ago, if we're honest? What did you take away from that show floor and all the conversations that you've had?

[00:07:06] Well, I think work happens on the endpoint where the employee is touching the technology, the laptop, the VDI, the mobile device. And that's what DEX is really all about, right, is understanding how your employees are consuming the technology services that you provide. We've been very good in technology, sort of monitoring the infrastructure layer or the cloud layer.

[00:07:35] And we've done a great job there. Now, I think we're shifting towards how is the employee actually consuming those technology services? And where's the friction? But we've done a great job in keeping everything green at the infrastructure layer, but we've still got thousands and thousands of issues coming in from employees at the support desk.

[00:08:02] So DEX is really about creating that autonomous digital workplace. And that's why I feel that it's becoming a business level priorities, because there's a real, real push with AI that's enhanced this to drive productivity for the employee and manage your IT a little bit differently to how we were before. Not just wait for people to tell us there's a problem.

[00:08:29] We should be proactive going out there, finding issues in the environment and creating this sort of stable, autonomous workplace. Employees expect that now, I think. I think that's an expectation of the employee as well. Yeah, I completely agree with you, and especially around measurement as well and doing things differently.

[00:08:51] Because if productivity, AI readiness, workforce resilience and cost efficiency, if all these things are increasingly tied to the digital workplace, should CIOs stop measuring success through uptime, tickets and technology adoption? And if things are moving that differently, what should they be measuring instead if they really want to demonstrate the business impact of this technology? Yeah, I mean, it's a really interesting point, right?

[00:09:18] Because we're measuring on the ticket front, say. Yeah. If we're measuring a decrease in tickets, that doesn't necessarily mean we've done a good job. Yeah. We know that 50% of employees no longer raise tickets. And so I think we need to look at things completely differently for a different lens. Productivity is obviously a key metric there. Friction. How much friction have we removed from the environment?

[00:09:47] How many issues have we removed from the environment? Is another metric that we're seeing customers use. With AI now, we have an authentic AI solution. With our authentic AI solution, we can be much more granular in terms of the productivity and time savings that we're giving back to the employee from interacting with our technology.

[00:10:14] And I also feel that we have to tie this back to bigger business outcomes or more important outcomes. The days of IT just saying, oh, we've put the app on the desktop and we can walk away are over. Or, you know, we've deployed the fix or we've deployed something to the CRM. How does that tie to improving sales productivity or increasing time to close?

[00:10:43] That's where we need to create that link between the technology and the actual outcome for the business. 100%. And I think another reason many AI projects stall now is that companies are inserting new technology into fragmented data environments and insufficient workflows that were built for an entirely different era. So how can leaders better identify whether AI is genuinely improving a process or simply automating problems that were already there in the first place?

[00:11:13] Yeah, I mean, it comes back to really that baseline piece, I think, that we spoke about. And if I look at and think about it in the digital workplace specifically, our Agentsic AI solution can really change how you manage incident and request management from an employee perspective.

[00:11:37] So the days of an employee going to a portal or calling the service desk because they have an issue are now over. They can speak to our Agentsic AI solution, Spark, and that can resolve the issues autonomously for the employee.

[00:11:55] So when we work with customers now, we understand the metrics, we understand the time for a ticket to close, the average time for a ticket for the employee. We understand ticket volumes. We understand the impact that has on productivity for IT. And then we can measure all three of those different currencies and the cost of a ticket.

[00:12:19] And then we can start to align those different currencies of value back to that same process. And then we know exactly how we're improving the incident and the request process within that organization. So. Hopefully that answers your question, Neil. Yeah. And I just wanted to point that out is that when we talk about currency of we spoke about currency of value earlier. Yeah.

[00:12:45] There's different, going back to that point, different currencies for different stakeholders within that process, right? The help desk manager might just be focused on ticket volumes, whereas someone more senior is focused on a CIO is focused on the cost element. Someone, a line of business is focused on the productivity piece of their employees. And it's such a great point, because I think there are potential blind spots there with enterprise AI.

[00:13:15] Dashboards might tell leaders that licenses have been deployed and tools are being used and everyone's patting themselves on the back, but not whether employees are struggling or creating workarounds or losing productivity. So I'm curious from what you're seeing, are there any other signals that a leader listening or an organization should be looking out for to understand the actual real employee experience of AI? Yeah. Yeah.

[00:13:39] So our observability platform can provide you full observability of every AI tool used in the environment. So if you've bought, I know 20,000 copilot licenses, we can tell you whether they're actually being used, which I think is, which is important.

[00:14:01] I mean, that doesn't tell you the outcome, but it's a starting point to say, you know, half of the estate aren't actually using the tool at all. We can then identify through that data where you have your champions, where you have your laggards, how deep the usage is within those applications. AI tools, sorry.

[00:14:25] And we also have a functionality now where we could, where we had this for a while, where you can interact with the employee and guide the employee through how to use these tools more effectively. So in the moment. So it's not, when we think about training, we're not just taking somebody out of their role, sticking them in a room for a day, going through the product, expecting them to be experts when they leave.

[00:14:54] And then that's it. We can give them in the moment guidance and we can gather their feedback as well, in terms of where they're seeing value from the tool, where they're not, where the challenges are, where the friction is and how much time they feel these new AI tools are actually saving for them.

[00:15:16] And then we can obviously have that data and we can look at across the line of business, business units, the wider business, where our gaps are, what parts of the business are using it well. Yeah. And it has been traditionally difficult with AI adoption being increasingly uneven, whether it's somebody listening in a large or small business or different teams and employees within the same organization.

[00:15:43] So what does an emerging employee experience gap mean for, let's say, workforce performance overall? And how can leaders prevent AI from creating yet another divide between employees who benefit from it and those who might risk getting left behind? We've touched on a bit of this already. I was wondering if there was anything else there that you could add? I think this comes back to like the persona piece. Yeah. In my view.

[00:16:12] We've got so many different generations now in the workforce as well. So like naturally you're going to have your younger generations are going to be more accustomed to the new technology. They're going to adopt it faster. We've also got people in the workforce which have been used to working in a way for so long that changing those behaviors can be challenging as well.

[00:16:36] So fundamentally, I think all of this is like in any change or transformation, the people element is the most important part from my perspective. The technology really is the easy part. Getting people to change the way they work is the challenging part. So I think that you have to have data. You have to have data on this like to show where the challenges are. You can't just go in blind.

[00:17:05] And I think with people as well, we really have to focus on the personal value. Like what is the personal value to the employee to adopt these new tools and adopt these new ways of working? Sometimes we focus too much on the business element, like the business. What's the value for the business? But what is the actual value for the people in your business using these tools and learning these new skills and ways of working?

[00:17:34] And I think that can really help drive that change and transformation that's needed. Yeah, and I think that culture element is often the most difficult, removing that mindset of, well, hey, we've always done things this way and making it easier for people to change. And I'd love to try and give people listening an actionable takeaway here.

[00:17:54] If we bring all this together, if a tech leader listening wants to move from that scattered AI experimentation that we mentioned earlier in our conversation and head towards that measurable business outcome that we've talked about and improving the experience for their employees, most importantly. Anything you'd advise them to examine across their people, their processes, data, and overall digital employee experience? If you are entering an organization, what are the first things you'd tell them to look out for?

[00:18:24] Well, the first thing I would say is make sure you have data and visibility of what you're putting out into your environment. Just from an employee perspective, but also from a risk and governance perspective. But you don't know what you don't know.

[00:18:42] So you need to make sure that employees aren't putting sensitive data into places they shouldn't and guiding and coaching employees to use the corporate branded tools. I would also say that maybe don't go big bang straight away. Start small. You know, showcase the value. Understand what you're actually trying to improve.

[00:19:11] And I would also say that, you know, you need to make sure that you've got your employees on board with what you're doing. The communication, the change element is so, so important. The technology really, getting the technology to the employee now is actually, it's become so much easier. That is, that is genuinely the easy part.

[00:19:38] It's getting the people aligned, getting the people on board and thinking about it from a persona based approach. You know, not every, like you need, you need different tools for different outcomes. So I think that element is really, really key. Like if you look at it from a digital workplace perspective now, there's not many workplaces you go where we give everybody the same device.

[00:20:08] It's exactly the same scenario there is that not every model produces the same outcome. So we have to think about how different business units do things in, in different ways. And the other, the other piece there just to add are now being asked to work a lot more. What I see is to work a lot more closely across the business. They're not, we're not just the people that sit in the corner and fix things.

[00:20:37] There's a, there's an element of working more closely with different areas of the business to understand the, especially in an AI first world. I think. Yeah, I completely agree. And I think that is a powerful moment to end on. And for people listening, there's a great conversation start. And I think you've turned quite a few light bulb moments on there.

[00:20:59] So where is the best place for anyone listening to find you or your team online and find out more information about next thing and anything we talked about today? Where should I send people? Yeah. So they can go to next thing.com. You'll have all the information there. They can see our products are out that we have for AI observability, AI drive. You can find me on LinkedIn. We have, we do have our own podcast.

[00:21:27] If people want to go and listen to the deck show and understand more about decks that is available on Spotify on more normal, normal platform, normal podcasting platforms. Yeah. And that's, that's about it really. I mean, thank you very much for having me. It's been, it's been great. Well, thank you so much for joining me today. I'll be including links to absolutely everything, including the podcast decks. I will be checking that out.

[00:21:57] And we did cover a lot today. Yes. Around AI and technology and decks and the workplace experience, et cetera. But I think the big message that is changing the mindset, making it valuable to your people, bringing the employees along for the ride. That is a big key message. And I'd love people listening to feedback. Let me know your story, your insights, how you're dealing with everything over at techtalksnetwork.com. But more than anything, just a big thank you to you, Scott, for starting this conversation today. Thank you. No worries. Thanks for having me.

[00:22:28] I think Scott's argument today leaves technology leaders with a useful test before approving yet another AI rollout. Define the outcome. Measure the starting point. Understand the personal value for the employee that is expected to change how they work. A deployed license proves distribution, yes, but it says very little about productivity, frustration, workarounds, or whether the chosen tool actually fits the person using it.

[00:22:57] I really love Scott's warning there that fewer support tickets in a department can create false confidence when many employees have simply stopped reporting the same problems. And that is where digital employee experience data and AI observability could expose the gap between a green dashboard and a difficult working day. So thank you to Scott for joining me and shining a light on this today.

[00:23:24] But over to you, what evidence would convince you that AI is improving work across your organization? As always, techtalksnetwork.com. I'd love to hear from you. Let me know. And if you want to come on the podcast, also give me a shout too. But that's it for today. So thank you for listening as always. Bye for now.