Measuring AI by Business Results Instead of Adoption With Domino Data Lab

[00:00:00] The leading issue of agentic AI in businesses right now is ensuring agents act with compliance guidelines. And Denodo applies guardrails across your entire data estate. By aligning your company's data infrastructure under one system, these guardrails perform consistently across your platform. So start scaling your business and start with Denodo. Simply visit denodo.com to learn more.

[00:00:31] Is your AI program creating business value? Or are you simply counting how many employees have opened the software? Well, my guest today is Thomas Robinson, better known as T-Rob. And he's going to join me today to discuss his role as the CEO of Domino Data Lab. And discuss why adoption can be a terrible measure of AI success.

[00:00:55] And after 10 years with the company, including time as COO, he believes enterprises must work backward from revenue, savings, risk and other business outcomes. So today we're going to explore why governance can help teams reach production faster. Why companies need different forms of AI for different problems. And why domain experts are becoming even more valuable as technology improves.

[00:01:23] And T-Rob will also explain the growing interest in AI sovereignty. And why the system surrounding a model could matter far more than the model itself. Enough for me. Let me introduce you to him now. So thank you for joining me on the podcast today. Can you tell everyone listening a little about who you are and what you do? Thanks for having me, Neil. I'm Thomas Robinson. People call me T-Rob. I'm the CEO at Domino.

[00:01:51] And Domino builds a platform and provides solutions for mission-critical AI in the enterprise. We help organizations run demanding, sophisticated AI systems in high-consequence environments, making decisions at the core of the business, if you will. T-Rob, you've got the coolest name I've spoken to on here, let me tell you. But, I mean, you've moved from chief operating officer to chief executive officer at Domino.

[00:02:21] That's something that I picked up in my research quite early on on you. So what did you learn inside the business that maybe shaped your priorities for the role? It must have been quite a journey, but tell me more about that. Yeah. So I've been at Domino 10 years. And before Domino, I spent time at a large financial services institution building technology that is very akin to what many of our financial services customers do today. Here's what I've learned at Domino.

[00:02:50] So we are at a very crucial moment right now with AI. And I have a little bit of a contrarian take. I think the most important thing for companies that are focused on AI is to think first about their talent. And so I'm working on doing the same. There is no magic in AI that allows it to go run roughshod over your enterprise and suddenly solve all the problems. If there was, I think we'd all be on vacation at the beach somewhere.

[00:03:19] I think we've learned pretty quick that it is not that easy to do. The value is there. The value is there. But it still takes people at a lot of different points in the process to go implement this technology in a high-quality way. And so that's where my primary focus is now that I'm in this role.

[00:03:37] And the biggest thing I'm trying to do is bring great talent to our organization and then have that talent share with our customers what Domino has learned over the past decade or so about what it means to do mission-critical AI. And talking about what you've learned there along your journey, I think many companies have invested heavily in data science and AI platforms, et cetera. But many still struggle to move those models into production out of pilot phase.

[00:04:06] Where is it that that process typically breaks down? What have you seen there? Yeah. You know, I think we're in a new version of the same problem that we've seen in the past. Everybody has been quoting stats out of pick your favorite study. I went to MIT, so I like the MIT one that says something like 95% of pilots fail. For those of us who have been in the data science world for a long time, that totally makes sense. I think about our customers who are in the pharmaceutical business.

[00:04:36] Imagine if 50% of the new drugs you discovered actually worked out and got all the way through clinical trials. Like, we wouldn't have a problem with health care at all. Like, there is so much research and care that needs to go into building sophisticated AI models and the application of those in the enterprise. And it's a challenging thing.

[00:04:57] I think what's been going on recently, though, just in the deja vu we're going through, this particular flavor of AI, generative AI, and large language models, I think have captivated a lot of people. And more than just the nerds, right? Like, we have captivated everybody. Probably yourself, people who are in journalism or people who are doing media production. It's the first time that AI has really impacted sort of everybody globally.

[00:05:22] So, I think there's a lot of anticipation and a lot of hope for what AI can do based on what people see, you know, using it in their day-to-day jobs. But the difference between a tool for an end user and something that's making a critical, sophisticated decision at the core of a business is a really wide gulf. And so, that's the challenge. How do you go from technology to the application? And, you know, we have a couple of takes on that.

[00:05:52] We think the first part is governance is critically important because as you step up the value, you are also stepping up the risk, right? So, important decisions carry more risk and they carry more value. And those things go hand in hand. Very different than figuring out, you know, what to write in an email versus how you decide whether to underwrite a mortgage or how you decide which candidate therapy to move to the next stage in your clinical trial.

[00:06:21] Those are very different things. And so, the level of governance and care around how you pick that technology is really critically important. You know, I think the other thing that's going on as well is folks are – I call it this hammer-to-nail situation. Everybody's thinking, oh, we'll just use this great new technology. And, you know, there's a diverse set of different types of problems that are out there.

[00:06:47] And our core belief is it takes a whole variety of different techniques to actually go apply to those situations. LLMs, generative AI, they exist in a certain spot. They are very good at creating new things. I wouldn't say novel, but creating new things, right? They're still statistical-based. They have some issues or problems with reasoning and logic that makes them potentially erroneous.

[00:07:13] So, you've got to figure out how to match that particular value that you need to what the problem is. And, you know, something like a statistical model is not all the rage today in the public consciousness. But those are really important for particular decisions. You know, we think that finding the right tool for the job, having the governance in place for the critical decision-making, those are imperative in getting the real value here. Yeah, I completely agree.

[00:07:42] And you made some great points there because the arrival of generative AI has increased pressure on leaders to show results and show those results very quickly. So, how can teams move faster but without weakening governance, security, or trust? We've kind of been here a decade ago with move fast and break things. I think we know that's not the answer. So, what can they do differently? I was just in California last week, and I stayed by the Meta headquarters.

[00:08:10] And I was reminded that Meta changed from move fast and break things to move fast on stable infrastructure, which doesn't quite have the same ring as break things. But, you know, they updated that saying, I think, gosh, it must have been five or so years ago at this point. And I think in that is a really wise advice for folks in the enterprise. So, you know, first and foremost, we think about governance not as a limiting factor or something that's supposed to slow things down.

[00:08:39] We think about governance as something that helps you speed up. And if you build in the appropriate governance, it allows you to progress your experimentation to a production place quite a bit faster. If you live in a world where all you're doing is experiments and all you're doing is trying novel things without thinking about what's required at the end of the day to get those into a production context, we see that as a problem.

[00:09:07] So, you know, our product and the way that we think about it is putting governance alongside the development efforts. And the idea being that, I love this analogy, you know, if you're building, if you're constructing a building, along the way you have people who come in and check and make sure that it's being engineered correctly and things are built in the right way, right? And then you end up with a building.

[00:09:30] You don't start, once the building is built, looking at what has been built and trying to validate that it's the well-architected and well-constructed and all that. You do it along the way. And we think the same thing is true for AI. So, in our product, you know, we think about three layers of governance. We have a policy engine that helps define what should you be looking at, what should you be checking, what's the right sort of things to evaluate along the way.

[00:09:55] The second is we also have capabilities for once models get into production, for monitoring and tracing AI systems to, you know, have continuous oversight and make sure that things don't go wrong. Inevitably, when things get in the world, the world changes or people do things that you did not expect. And so, being ready to digest and adjust your systems is critically important. And we're still big believers in human in the loop. Maybe you sense a theme here, talent, humans, people.

[00:10:23] I think humans in the loop are critically important. And especially as we are learning to work with some of the generative AI models and trying to understand, are they working or is it an illusion? I think, you know, there's a little bit of that going on. So, you know, I think that's a lot of what can really help folks improve. It's put the governance alongside, make sure you still have humans in the loop and have continual monitoring and tracing to make sure things are working the way that you expect.

[00:10:50] I always try and give people listening a valuable takeaway. So, for anybody listening, what does an enterprise really need in place before it can start scaling AI from just a small group of specialists into teams right across the business? Any big takeaways here that you'd pass on to people listening? Got a couple of things. First, I'll quibble a little bit with the framing of the question.

[00:11:14] I'm fond of saying that, you know, focusing on the AI rather than on your business is the wrong approach. Yeah. I think business leaders have to first look at what is our company? What are we doing? What are the few core metrics that are important? And work backwards from that. Because if you start with, hey, I got this cool tool. What should I go do with it? I think it's a recipe for disaster or maybe a recipe for wasting a lot of time.

[00:11:40] You know, we encourage our customers, be it in the line of business or thinking about your business overall. For a pharmaceutical customer, for instance, their couple of key metrics are how many new drugs are they bringing into the pipeline? How much are they driving revenue? How are they dealing with risk? You know, they have a few key things that they're focused on as a business. And all else is basically support of operating that business. So working backwards from there, I think, is the most important thing that we recommend people do.

[00:12:09] The second relates a little bit to this hammer-to-nail thing that we talked about before. We recommend that you leverage all of AI, not one particular tool. So if you start with, again, just to this mindset, if you start with tool-to-problem, that's not the right approach. You need to look at the full corpus of tools that exist. You know, sometimes that's computer vision algorithms. Sometimes that's statistical models. Sometimes that's simple rule-based systems.

[00:12:36] And integrating those together is where you get the real value because you get a blend of the right risk tolerance, the right approaches into a more robust system. And then, you know, third, we've seen a lot of companies make the mistake of thinking people can be entirely replaced with AI. I think this has settled down a little bit. I think there's been a dose of reality as we sort of move through the phases of this AI cycle.

[00:13:04] You know, starting out, I remember companies were firing their best experts or deciding, you know, CEOs were bringing down their headcount in anticipation. I don't think they were ready yet, but in anticipation of resolving things with AI. And we think that in a world where there is more artificial intelligence, the domain experts you have are even more critical. Those are the people who actually understand what it takes for that last mile.

[00:13:33] They're the ones who understand your business over time through different regimes. They're the ones who have the wisdom that the models don't have when they go in and look at the data. And so those are kind of the three things I would say. One, start with the business problem, not the technology. Think about all of AI, not just one type of AI. And third, hold on and empower your domain experts rather than trying to push them aside.

[00:14:02] And something else I wanted to highlight today is that at Domino, you work with highly regulated organizations. And I'm curious on that side of things, how do their requirements change the way AI platforms must be designed and operated? Are there big differences there? I like to start, the fact that a regulator exists for a given company or a business is often a reflection of how important or critical that organization is and the things they do, right?

[00:14:31] It has the, they're often in industries where there's the potential to cause a lot of harm. That's why we have a regulator, right? And so, you know, for me, that is exactly the zone that we love to be working in is where there is high consequences from the decisions that are made. And so from day one, we've been building Domino for those sorts of needs, you know, not for the low risk, broad end user, but for the really critical places.

[00:14:59] And in fact, we actually get to work with some regulators, so helping them build their technology to do the regulation. So I find that particularly exciting. You know, so the pieces that we've put in here, we've talked a little bit about governance. That's been something built into our platform for the longest time. Being able to look at reproducibility is, and that's something that's critically important for regulators. They want to see, they want you to prove your work, right? It's like doing your homework. They want to show, want to see how have you proved out that thing works effectively?

[00:15:29] What did you try? What risks did you look at? And so we've always built the platform to capture that work along the way. I think one people, one thing people don't realize about sort of a scientific or research type endeavor, the things that you prove work are equally as important as what you prove doesn't work, right? And having that whole corpus of knowledge along the way. So we built that into Domino over time. The second piece is, you know, these organizations have existing controls in their enterprise.

[00:15:59] They have access controls. They have audit requirements and systems for doing all of that. And so we've always integrated in the entire ecosystem of an enterprise rather than making them reinvent a lot of their existing controls. So yeah, those are a couple of things that we've done. I think the things are a little bit different about regulated businesses.

[00:16:19] And also, as companies continuously adopt open source models, new commercial models, and their own internal systems, how should they avoid becoming locked into one approach? One of the reasons I asked that is a few moments ago you were talking about it should never be the technology. You know, it should always be the business problem. But tell me more about them. Yeah, I was thinking a little bit about this. And I think there's a great lesson from the cloud computing era, right?

[00:16:47] So I think starting, what, maybe 15 years ago, but in earnest over the past 10 years or so, folks have been moving to the cloud. And I think one of the mistakes that a lot of enterprise IT organizations made was picking a particular cloud. And, you know, what happened was they built on top of a lot of the core services increasing amounts of their own technology and essentially got locked into using a particular cloud.

[00:17:16] And that has a lot of issues with it. One is your ability to negotiate for a great price. Yes. Importantly, especially in Europe, you know, with things like Dora, we see that not being single sourced on a particular vendor for an operational risk issue is critically important. And so I think there's a lot of lessons there that can be adapted going into the AI era.

[00:17:40] If you are building on one vendor and you are trying to use the probability that that vendor is going to be the right one in 10 years and the probability that you only use one vendor seems incredibly low. That's sort of like the broadest perspective first is when you're engineering these systems, you need to be looking for something that allows you to switch out different types of models.

[00:18:04] We, in our product, have a philosophy of building an agnostic and open ecosystem. We've always worked with enterprises and you don't get to tell them which database to use. They have all of them already. You know, the same thing goes for models. So we've spent time building the same open ecosystem for model providers and LLMs. The next thing that I'd say, though, I think is a little bit unique about open source models versus maybe some of the foundation model providers.

[00:18:34] Models are getting commoditized very quickly. And, you know, we see that many of the open weight models are very much catching up to the frontier. And, in fact, for a lot of use cases, you can be very far off the frontier for a lot of basic things, especially when you build AI systems that leverage the right pieces. They don't all need to be the bleeding edge, most expensive frontier model.

[00:18:58] But I think if I had one piece of advice for folks out there, the competitive moat today is the harness, not the model. So I think what you're seeing in the early phase of adoption is the frontier models actually have a really great harness in that, you know, they've built a product that makes it easy to adopt. It works sort of pretty well streamlined. But very quickly, other folks are learning how to build sovereign AI stacks.

[00:19:26] They are learning how to build the appropriate harnesses around those models. And I think we're about to hit a Cambrian explosion, if you will, of using different models with different harnesses in the enterprise. And you did mention at the beginning of our conversation that famous or infamous MIT study about the 95% of AI projects failing. There was a lot of hype around those results.

[00:19:52] And it did push a lot of businesses back towards return on investment business value, which is a great thing, of course. So what should business leaders be measuring to tell whether one of those AI programs is delivering useful outcomes rather than just producing an impressive collection of experiments and shiny demos? The first thing I love to tell people, and I'll tell anybody who will listen, is consumption or adoption is a terrible metric for success, right?

[00:20:22] I think I mentioned we're kind of early, right, in this phase. And I think a lot of executives were looking at how many people are using the technology. And the problem is that's not where return on investment comes from. That's a great measure of cost. But if you imagine, if you rewind to a previous era and we went through how many people in the organization have a desktop computer or whatever, not a really good measure for whether the company became ultimately effective or not.

[00:20:50] You know, the first spot to always start is, again, back on those core metrics in the business. That is the thing you want to impact. That is where your return comes from. And tracking the investment to particular projects where you are working on those fundamental metrics is the most important piece.

[00:21:09] Related to that, when you – in an individual context, if people are using these desktop tools, you have the ability to maybe impact productivity 5%, maybe 10%, 20%. But that productivity does not accrete to the total business. It is not that you get – because everybody is 10% more efficient, your business is overall 10% more efficient. It doesn't work that way.

[00:21:32] The individual person is not – how everybody works together, how the process works, how they communicate with each other, that's really what stops us from just multiplying that productivity up to the top line. And so when you look at your critical business process that makes particular decisions, how much more revenue, what are the savings?

[00:21:56] Those are the things that can actually drive a material return on investment and where you get sort of the outsized value. But in order to do that, again, it comes hand in hand with the risk, right? The higher consequence, higher risk. And so you have to be much, much, much more careful. I think, thankfully, the investment required is a lot more around designing the system than it is necessarily pure tokens.

[00:22:20] And I think there'll be a point where we get massive multiplicative impact from simpler AI systems that can actually go impact those big metrics. And by the way, you know, we call that thing that's going on the great AI rotation. It is going from individual users to core of the business. And we think we're in a moment where there's going to be a big explosion of wanting to really drive the core of the business.

[00:22:47] And that's a lot harder challenge than simply getting a tool to somebody's desktop. Thankfully, we've been doing that for 10 years here. And we've been really helping organizations do it in their innovation or in their core operational processes for a long time. And as we record this episode today, we can reflect on how 2026 has been all around agentic AI and agents, etc.

[00:23:12] We've now got one eye around the corner at 2027 trying to work out where this is going to go next. So as Domino enters your next chapter under your leadership, where are customers asking the company to improve or move faster? Are there any trends in the kind of things that they're coming to you and asking for your help with? Yeah. One of the things we're reacting to is back to this point of people.

[00:23:36] And, you know, we see a lot of organizations who are saying, help me change my process, not help me or give me pure technology. And so, you know, we're building a forward deployed engineering team that can go out and understand how to work with customers, understand what's actually going on in their business and help them build the AI systems that can be used to drive improvement.

[00:24:05] And the trend that's going on, I mean, one of the big benefits of the generative AI movement has been coding, right? Coding assistants have really helped compress the amount of time it takes to code. It used to be that you hired software engineers to type into a computer more code. I mean, I'm being a little pejorative. I used to be a software engineer, so I can do it. You know, the agents have really taken that away. But what has gone into its place is the need to do really good specking of what you want.

[00:24:35] Everybody knows they've tried to prompt an AI to do exactly the thing they want. It's challenging. It can be challenging, and you can waste a lot of effort there. So really being thoughtful about how you spec. And then on the testing and then the integration side and the governance side, as we've talked about, those are still critical things that require people to look at the system and say, what's the right way to do this? What's the right thing to build?

[00:24:57] And so, you know, that's where we're pivoting as an organization is to have more people at the ready who are experts, who are AI native, who are experts in building these systems and can come in and take the technology to apply it to the specific process. I'd say, I think the second big thing that's going on, though, is AI sovereignty.

[00:25:17] I think very quickly, CIOs are realizing, oh boy, I am sending a lot of potentially sensitive information out the door every time my organization prompts a foundation model. And, you know, there's a real risk of what we call enterprise distillation. So, you know, taking out sensitive IP or competitive information from organizations and sort of spreading that across, you know, the broad market.

[00:25:44] I don't think there's espionage at the level of steal from here and give to there. But, you know, the thing that can happen is a lot of your novel research or competitive advantage can be described and then built into the model in the future. Anonymization isn't enough to solve that problem. It actually needs to be guarded against. So we're hearing AI sovereignty being top of mind for folks. And I mentioned a little bit before, I think the mode is the harness.

[00:26:10] And so, you know, we're thinking about how to make the models valuable as a full stack for sovereign AI. So give me an estate where I get the same quality that I would get from a frontier model provider, but make it so that it can be completely within my enterprise and I can feel like I have the right guardrails and security around it. So exciting times ahead. And I'm curious, is there anything else that excites you as CEO?

[00:26:37] Make sure you want to jump out of bed in the morning when you look forward at this. So much unprecedented change and opportunity there. Anything in particular that's exciting you at the moment as you look ahead? I'm super excited by the missions our customers are on. I mean, I get to go see some of the coolest things. I was just looking back this weekend at some of the things we've done. We're helping deal with contested waters around the globe for defense organizations.

[00:27:04] We're helping pharma companies who are on the cutting edge of developing some of the most breakthrough drugs. And, you know, we're helping make sure that financial markets operate effectively and fairly and securely. And so, man, that's what gets me out of bed in the morning is I look at these challenges and I say there's no better place to be trying to apply AI than at the vanguard of what we're doing for the world. And so I get really excited about that.

[00:27:32] I think that's a powerful moment to end on. And for anyone listening that feels somewhat inspired today, maybe they want to continue a conversation with you or your team or just find out more information about your work and how you're helping customers. Where should they go? Domino.ai. It's the easiest way. It's our website. We also host a conference every year. We do one in London where most of our European customers and prospects come out. And then you can always find us on LinkedIn, Domino Data Lab. Awesome.

[00:28:01] I'll have links to everything that you said there. And I encourage people listening to check you guys out. And also, I think your passion for the topic really shines through today. And also, I think that journey that you've been on has shaped you in such an incredible way as well. So I wish you the best of luck for the rest of the year. Hopefully we can talk with you again next year, see how things are continuing to evolve. But thank you for your time today. Thanks so much, Neil.

[00:28:25] One of the things I loved about T-Rob's argument today was his refusal to treat AI usage as proof of business progress. At last, we're waking up to this. I mean, a thousand employees opening an AI assistant, yeah, that might produce an impressive dashboard. But it tells leaders very little about revenue, savings, risks or better decisions being made. And I think his advice to begin with the business outcome, choose the appropriate combination of technologies and then build governance through development.

[00:28:55] Retain people who understand how the company actually works. This stuff is pure gold. And that final point as well about when automation is often discussed as a substitute for expertise. That is something that we need to overcome too. But over to you, what does your organisation currently measure when deciding whether an AI investment has delivered a worthwhile result? I want to hear all your stories. Good, bad and maybe the ugly too. And you can get me at techtalksnetwork.com.

[00:29:24] Love to hear from you. But I'm afraid I've taken up far too much of your time again. If you're still wanting more, I'll be back again bright and early tomorrow. But thanks for listening as always. Bye for now.