Giving Enterprise AI Agents the Business Context They Need With SAP

[00:00:04] - [Speaker 0]
What does an AI agent need to genuinely understand before you trust it to complete work right across your business? An assistant might be able to help with an individual task, but coordinating procurement, finance and supply chain activity will possibly bring a different set of questions and a certain amount of caution. And the reason for that is many will be thinking, does it understand the state of an order? Can it access the right information? But who controls what it can do?

[00:00:38] - [Speaker 0]
And how do you measure whether the result was worth the cost? Well, today, I'm delighted to be welcoming back Erfan Khan back onto the Tech Talks Daily podcast. He is the chief product officer for SAP Data and Analytics, and our conversation this time will be looking at the business context behind reliable enterprise AI. Because we're gonna discuss why problems often appear when a pilot reaches operational scale. We'll discuss all this along with the role of SAP Business Data Cloud and the choices leaders are facing right now when connecting agents to the systems that their organizations already heavily depend on.

[00:01:21] - [Speaker 0]
But enough for me. Join me in welcoming back Erfan to the show now. So a massive warm welcome back to the show. For anyone that missed our previous conversation, can you tell them a little about who you are and what you do?

[00:01:36] - [Speaker 1]
Neil, great to be here again. Erfan Khan here from SAP. I'm the chief product officer for SAP's data and analytics organization. So I've been here around about eighteen years, came via an acquisition from Sybase, and I've been pretty much in data and analytics my entire working career, which is too many years for me to to agree or even to acknowledge at this stage.

[00:01:57] - [Speaker 0]
Love it. And as you know, every day on this podcast, I try and demystify a different area that people keep hearing about. There's a lot of buzzwords out there at the moment. And when SAP talks about the autonomous enterprise, it's something we hear a lot about right now. What does that mean in the daily operation of a, I don't know, a finance, supply chain, or even HR team?

[00:02:20] - [Speaker 0]
What does that mean on the ground?

[00:02:21] - [Speaker 1]
That's a great question. I mean, autonomous enterprise is playing on on words to some extent. I mean, the word autonomous means running with potentially a human in the loop, maybe no human in the loop. And across the enterprise, naturally, there's a big distinction when anything runs within an enterprise landscape. You have to honor and manage all of the enterprise grade requirements, whether that's disaster disaster recovery, availability, all the way through governance, security and all the other aspects that you need to cover off.

[00:02:50] - [Speaker 1]
And when we announced, albeit from an architectural point of view, the autonomous enterprise earlier in the year, our intent was very, very clear. Take, for example, the domain of finance. In most customer environments, you tend to have multiple stakeholders that all contribute towards finance. I mean, it could be something from SAP, from a general ledger. It could be maybe data coming out of a completely different system or a different environment.

[00:03:16] - [Speaker 1]
And in order for the one end to end foundation to be delivered to an end customer, and they wanna have that one single experience where they're not just looking at task automation. They wanna be able to string all of these discrete activities together. It needs to operate as one autonomous enterprise, not disparate or discrete little aspects that have to then be coordinated. This brings everything together under one single foundation, and that's essentially what we've been working towards ever since the announcements back in May.

[00:03:43] - [Speaker 0]
And I've been to a lot of tech conferences this year, and one of the things that I've learned and seen firsthand is how many companies are already using AI assistance for individual tasks. More recently, I was at a conference last week in Orlando, and they were talking about digital workers, and we're talking about teams of agents now. But what do you think has to change before these coordinated agents can complete work across several different business systems?

[00:04:10] - [Speaker 1]
The the fundamental gap that still needs to be addressed, and we are feverishly trying to address this, is the context gap. Yeah. If you have an agent and it operates within the domain of, say, procurement, and you have another agent that operates within maybe supply chain, it's pretty pointless with those two agents just sending each other potential instructions or even coordination attempts if they don't know what the state of a purchase order is, for example, or the order on hand maybe or the inventory level. So this is why the foundation of agent to agent or even for that matter, broad coordination of agentic workflows right now is un... Unfortunately, right now, it's not with all of the necessary requirements of context that are needed to be really able to drive the enterprise grade.

[00:04:53] - [Speaker 1]
So that's one fundamental problem that needs to be addressed is the context gap. The second one is around the trust and governance. I mean, you could imagine when you have a very well defined deterministic IT system, everything is curated based upon user access and control. Now if you have an agent that runs completely independently, you want to make sure that you manage the same level of governance and security, fine grain and coarse grain. So that's another element.

[00:05:18] - [Speaker 1]
So context gap and of course authorizations and making sure that the governance piece is there. And then the final piece is really is the time to market and time to value. Most customers are eager to get from experimentation to a full blown deployment, and I feel as if that's where the biggest challenge lies because you have to deal with the previous two points before you can get to the the actual outcome now.

[00:05:39] - [Speaker 0]
Yeah. And you mentioned a few challenges there, and I do repeatedly hear around problems that are typically around data quality, process design, and permissions. But I'm curious. When do these things usually become visible when an organization moves from outside of that AI pilot and into operational use? What what typically happens here?

[00:06:00] - [Speaker 0]
What have you seen?

[00:06:02] - [Speaker 1]
Most IT departments are pretty well governed, and hence, they're they're wholly curated environment that you tend to have, data lakes with data that is nicely curated and lands in a in a storage format of your choosing. And now answering the question, when do these problems materialize is really when you go from that POC to actually then let's run this at scale, be it in a in a departmental level experiment or even beyond or or, you know, maybe even beyond that. And I feel as if most of the organizations right now are very mindful about these topics, but it's not straightforward because it's almost as if you've gotta go back to the fundamentals. You can't really have a reliable AI without reliable data, and reliable data means that you have all of those things that you just mentioned covered off. Quality, making sure that you have the right governance, making sure that you have the right security, all those things need to be given as a strong foundation before AI could be successful.

[00:06:52] - [Speaker 0]
And to bring to life everything that we're talking about today and maybe make it resonate with people and what they're seeing inside their own organizations, are you able to share an example of maybe a less visible AI application that has improved a a real measurable business result without becoming a a headline grabbing demo or something we've seen in our news feeds? Because I think these are the kind of stories that that really do resonate with leaders.

[00:07:17] - [Speaker 1]
Yeah. I believe that the the most important parts of AI today are where you can generate a lot more automation around basic tasks, workflow types of things, approvals, making sure that you can achieve a much quicker time to market based upon certain conditions that need to be met. Those things may not be headline grabbing type of activities, but they are fundamentally at the core of really driving forward with AI. So I would kinda go along the lines of really task automation and workflow management.

[00:07:48] - [Speaker 0]
And there's been a a huge focus over the last eighteen months around return on investment on some of those expensive tech projects. And then the the old IT mantra of you can only improve what you measure, but, of course, knowing what to measure, what metrics to use, that is where it can prove to be quite challenging too. So for leaders listening, how should they be measuring whether their enterprise AI is is genuinely improving productivity, accuracy, speed, resilience, and so many other different areas rather than just generating more activity, more noise?

[00:08:21] - [Speaker 1]
I think that's a great question because there was a feverish set of activities that have underpinned the whole generative movement, generate more text, generate more content of any description. And that could well be a measure, right? But it's not the measure. And what we tend to find the majority of customers right now is they will have a high degree of expectation around AI and where they look at measures of success. I mean, at the top level, could argue and say, if you're the CFO, you'd wanna be able to break out your company earnings and report e...

[00:08:51] - [Speaker 1]
You know, EBITDA and how that was affected by AI or the initiatives around AI. That's maybe the macro. If you then go down a level, you could be either take any discipline. If it was in my realm of development, maybe you want to talk about developer productivity, which could be everything from vibe coding all the way through to, of course, using maybe more... Not just large language models, but also tabular models to be able to accelerate prediction or forecasting.

[00:09:14] - [Speaker 1]
So the measures actually do depend, and there has to be some kind of an intent based approach that you take here. In the most part, what customers are really looking for is that they don't want to just throw tokens at the problem, and then the tokenomics don't actually add up. I mean, you probably recall hearing Neil over the summer where... I think it was AWS. I could be wrong here, but AWS was eliminating certain sort of roles, only finding that the roles that they eliminated was now being met by AI and token consumption, which was like two x what the individual human being was doing.

[00:09:46] - [Speaker 1]
So that clearly isn't the best vantage point either. So I think measures are gonna be multi... There will be a multitude of measures, but the most important one is really the end to end productivity. And I I personally seen this, at least from a coding and development point of view, we are now fully incubating AI based native development and, of course, AI assisted development. An assisted development was a starting point.

[00:10:08] - [Speaker 1]
Native development, of is the pursuit of of of every single development organization to start with design all the way through to delivery and then operationalizing the way that the service is available. So you'd wanna go really from end to end along those... On those on those paradigms there.

[00:10:22] - [Speaker 0]
Yeah. You raised a great point there, especially around the tokenomics. And I've read a a few examples of businesses that have done that and thrown everything at AI, but I think by q two or q three, they were already paying more than they were to an individual. And are are you seeing teams beginning to recognize this and you can't use, or burn tokens for absolutely everything, and it's just certain tasks? How how are you seeing them get that balance right?

[00:10:48] - [Speaker 0]
Is it something you... You're seeing a lot or a fear of of those token costs?

[00:10:52] - [Speaker 1]
I mean, our own personal experience, and I'm sure I'm probably not in a minority here in the industry, is that you start off with the the desire for everybody to start becoming or working towards the AI native approach. And therefore, it's without guardrails. And then very quickly, when your token consumption gets to a point threshold a or threshold b, consider whichever one is that you you think you're at the moment, listeners are at, threshold a may will be, oh, we better take a look at this because we may be overstretching our boundaries in On top, consumptive type of credits that we probably were not prepared or were not really necessarily gearing up towards in support. And when you get to threshold B, it may well be, Okay, now we need to do something drastic. We cannot operate the same way because there's causality here.

[00:11:36] - [Speaker 1]
And if you look at the value that's been attributed versus the cost, I think it's debatable in some cases whether the cost is actually supplementing the the, you know, the value that that potentially could be there. So I feel as if most organizations are in their their own journey, we for one in SAP, we really let our developers experiment, but like many others that we're also now becoming much more driven by standards and taking a look at what tasks can benefit from consuming tokens versus those that probably will not benefit and becoming much more deliberate in the way that the AI is then put to work. And it can't just be that you use it for every single task because that in itself may not give you the return that you desire.

[00:12:15] - [Speaker 0]
Yeah. 100%, Lydia. And another big talking point right now is accountability. So when several AI agents contribute to a decision or a complete process, how should responsibility and audibility... How how should these things be assigned, especially if the result that comes from this is unexpected at best?

[00:12:36] - [Speaker 1]
You know, we have a quite a... An interesting, I believe, quite innovative way of looking at this, which is company memory. And what is company memory? It's an initiative that we are running within our process improvement process integration layer where if you think about a classic on a sales order process, you know that that process will take certain paths, twists or turns depending upon the time of the month, quarter, whenever it may be, but it operates within certain thresholds. And when you now run the risk of running all these different things with agents all discreetly chipping in, doing things, not not not necessarily necessarily contributing to the end outcome, but unnecessary for context, you have to be super careful that the outcomes themselves are not being heavily compromised.

[00:13:19] - [Speaker 1]
And this is now where company memory comes in its own. And imagine you've got maybe a classical sort of front office desk, right, where they manage procurement for for the organization. They they deal with customer orders, but at the same time, in the background, they do fulfillment based upon certain partners or partnerships. You'd wanna make sure that you manage these partnerships holistically. If you know, for example, that q four typically or Black Friday sales coming up, you have a means of being able to take advantage of the buying power across multiple partners and then passing that through to your consumers, you need to know that.

[00:13:53] - [Speaker 1]
Agents obviously will have no recollection of that. That's the humanistic side of things. So company memory really gives a means of agents to operate knowing what they should do, but equally importantly, what they shouldn't do. And I feel that's one of those key differentiation opportunities because if you own the process and you own the back end and you understand the operational systems, it's well within our grasp, be it through the autonomous enterprise and allowing that level of fluidity in terms of outcomes and agents participating, let's call them a swarm of agents, who all wanna interoperate, for them to operate within more of a governed and, at the same time, institutionally acceptable manner. This is one of the key areas that we're looking to investigate here.

[00:14:32] - [Speaker 0]
Your agents aren't producing accurate answers because they don't have a complete semantic understanding of your data, and Denodo is solving this and solving it through semantic consistency. Through semantic consistency, your agents can start making accurate predictions in real time. So see what else Denodo can do by visiting denodo.com to learn more. And I'm curious from all the conversations you're having and the customers you're talking with, what have you learned about employee trust when AI begins completing work rather than just recommending the next step? What what was the narrative like on the ground floor there?

[00:15:14] - [Speaker 0]
What are you seeing and hearing?

[00:15:16] - [Speaker 1]
Well, the the narrative is is actually quite nuanced. Yeah. Trust trust is is is not subjective, but it certainly can be to some extent what does it mean when you when you are outsourcing some level of of the workflow or pipelining based upon an agent. The majority of customers are paranoid, and rightfully so, but they don't want ungoverned agents to do certain things without understanding that there could be reputational risk for the organization. So typically, what happens is that you put much, much more AI governance and guardrails around the process.

[00:15:49] - [Speaker 1]
So trust at the employee level has got to be some checks and balances. It's not uncommon to look at your spreadsheet just to make sure that you knew what last night's actuals were before you make a budgetary decision even though the budget and the real time budget may be shared with you directly through the agent because you still have a degree of not mistrust, but you wanna get that validation. So I feel as if the trust element is dependent upon the usage and dependent upon the persona and the intent of the user, certain things will provide you be much more lapsidades. You say, fine. Okay.

[00:16:21] - [Speaker 1]
It's an approximate number. I'm good with that. You know, if you see, think about order entry and you think about inventory on hand, you'd want that to be relatively precise. Right? Relatively very precise for accuracy purposes.

[00:16:32] - [Speaker 1]
If it's not, then you're gonna end up fulfilling or potentially trying to fulfill an order that just wasn't achievable. That has reputational risk. At the same time, if you're now looking at looking at a historical sort of pipeline and you wanna look at conversion of a pipeline in the sales organization and you have some models and we've acquired some companies along the way, which I'm happy to... Love to share with you. We acquired a company called Priolabs, which gives us tabular approaches to being able to look at more of a narrow model in terms of looking at prediction and, of course, simulation and forecasting based upon much more in context learning, which means that you don't have to have a big training model based upon a frontier LLM before you can actually make a decision.

[00:17:10] - [Speaker 1]
So trust will go up if you can bring in more in context learning and more accessible type of forecasting and predictions within a workflow.

[00:17:19] - [Speaker 0]
So many great points there. And I always try and give people this thing a a valuable takeaway. And if we'd have a leader from an organization that have several pilots on the go at the moment but with very little measurable value under under increasing pressure to prove ROI improve, they're making a difference. What should they be changing during the next six to twelve months?

[00:17:39] - [Speaker 1]
The the role, Neil, of of platforms has been undeniable for for quite some time. But having a platform that really is geared towards allowing agents to be built, allowing data to be contextualized, then you can reason with the data, and then having a governance around that becomes an incredibly important end to end set of activities. And not to self promote here, but we have our business AI platform that does precisely that. It gives you the full end to end approach that you need to have. And also, given most customers' experiences of the past, they wanna have super open approaches to bring their own platform, bring their own data lake, bring their own table format, be Iceberg or other.

[00:18:16] - [Speaker 1]
And this is another key key key, you know, requirement that most customers have. They'd already be locked in within one vendor space, one LLM, one infrastructure provider. And these are the the hallmarks of future success because you only see this, and I think you and I can both testify to this, on a weekly basis, sometimes almost on a... Seems like on a on a on a daily basis. We see advancements being made by one lab compared to another, and optionality needs to be incredibly important.

[00:18:42] - [Speaker 1]
And this is exactly why I think most leaders right now are unrightly so cautious because they don't wanna back they don't wanna back the wrong horse, they certainly don't wanna go down a path which minimizes their scope of of innovation. So I think those are the real issues that most customers are feeling. And our intent through Business AI Platform, leading with openness, supplementing via Drenio, the acquisition that we really made around allowing federation and, of course, strong levels of integration across data lakes and Rails here that we recently acquired also to give the whole notion of being able to do unification of data, what we call unified data products. All these things will make a huge difference. And as I said, that's slightly self promoting, but I feel as if those are really the enterprise grade requirements that most customers are looking for right now.

[00:19:26] - [Speaker 0]
Yeah. I completely agree with you. And I think in the beginning, many leaders went all in on AI. There were so many questions, and I will cautiously suggest that many were distracted by obsessing over exactly which model they should be using. And, thankfully, that conversation is evolving now, and it was refreshing to see a video of you.

[00:19:46] - [Speaker 0]
I think it was on your LinkedIn feed a few months ago where you said enterprise AI needs more day... More than just data and models. It was great to hear you saying that. But on that side of things, we are talking about all things in the SAP ecosystem now. Can you expand on how the SAP Business Data Cloud powers the contextualized and reasoned layer of SAP Business AI platform to give agents that deeper understanding of how a business actually runs?

[00:20:15] - [Speaker 1]
I'm glad you asked that question because I was very eager to maybe address that, and thank you for asking the question so directly. You know, we have, at SAP, been custodians for customers, and and we're very privileged to have this ownership of core business processes and core operational systems for decades. Independent of whatever the innovations and there are many coming out of the Frontier Labs, we recognize that the transactional core is this transactional core of most organizations. So if we take a look at the evolution that's going on, they want to be able to contextualize that data and be able to use that in meaningful AI outcomes. Hence, the BDC introduction we contextualize now.

[00:20:54] - [Speaker 1]
What we're able to do is use BDC as a primary ingest of both SAP and non SAP data, which has now been supplemented with our latest acquired company called Dremio with the Dremio Lakehouse, which now means that you can actually go from zero copy share with any of the data platform or or or hyperscaler environments. That's a great value add if you're a Databricks customer, you're a Snowflake customer. You can continue to interoperate zero copy share within those environments. But now with also with the advent of Dremio, if a customer wants to bring their own data lake, they have an iceberg native, lakehouse that they built, and they wanna be able to allow federation or sequel analytical kind of workloads across those datasets without having to consolidate and move all the data. This is now where BDC has even an additional dynamic now.

[00:21:41] - [Speaker 1]
It allows us to interoperate both on the open side of storage, which is iceberg, and also on the catalog side, which is Polaris, right, which is a strong sort of driver of the standardization as a technical catalog across our industry now. So that's one area. And when now you talk about contextualizing data, if you have that infra kind of foundation where you can ingest data SAP, non SAP, represent that data as primary data products, and then now bring in the Raltio piece, right, which is another key element, and have the means of being able to unify data. Imagine you've got something like a call center. Call center is very, very complex because it spans multiple environments, SAP, non SAP.

[00:22:22] - [Speaker 1]
And one of the key deliveries that we have, around about 50 unified data products is what we're targeting for the end of the year, and a good example of that would be cost center where maybe the cost center lives across SAP, Salesforce, and maybe even Snowflake. And to have one single unified data product that has the context of a cost center semantically enriched... Now imagine that your AI and your accuracy through AI, if you can ask a question, a natural language question, in our case, using Dual, and you know that fundamentally there's going to be a high degree of accuracy. This is exactly where Business Data Cloud now contextualizes, allows the data to be reasoned with, and allows it to be served up directly through that Dual engagement layer, whether that's with dual work or dual studio, which we're looking to, of course, announce at TechEd and and giving customers a huge level of productivity gains from from that one end to end sort of foundation now.

[00:23:13] - [Speaker 0]
Well, we've covered so much in a short amount of time today, and I would love to include links to everything that we've discussed. So people listening can go straight over to techtalksnetwork.com. There'll be a blog post associated with this episode. I'll include all the links that I can find from everything you've mentioned. But, anyway, you'd like me to, point everyone listening?

[00:23:32] - [Speaker 1]
Yeah. I mean, Business AI Platform for SAP is now the new umbrella, Uber sort of categorization of of those three terms, the build, the govern, and all the way through the contextualizing, of course, and reasoning. And and I would strongly advise our listeners to take a look at that and you'll get that sort of end to end experience. And in each of these discrete areas, there's a ton of innovation work going on. I've not even covered off all the things because there's so much.

[00:23:56] - [Speaker 1]
I mean, on the HANA Cloud side, for example, we have this agent memory, which is there for long term persistency. So as agents run and they wanna preserve context, you can manage that context across multiple instances or or duration of the of the agent invocations. So there's a lot that we're doing, and I think that the TechEd news and announcements that we've just shared will also pave the way for broader levels of adoption for SAP and non SAP workloads running in the customer's environment. So we're super happy and at the same time, excited to see where we're at right now.

[00:24:26] - [Speaker 0]
I think the last time we spoke was something like two years ago, and it's phenomenal the amount of change that we've seen in that time. So we've gotta make a promise we won't leave it that long before we talk again because I think that pace of change is only gonna continue to ramp up. But more than anything, thank you for coming back on, talking about all this in a language everyone can understand and give people actionable takeaways. Really appreciate your time as always.

[00:24:49] - [Speaker 1]
It's a pleasure, Neil, and absolutely won't be two years until the next time we can... I I can assure you that.

[00:24:54] - [Speaker 0]
I think Erfan's example of procurement and supply chain agents was incredibly useful today because, yes, they can exchange instructions, but useful coordination, that depends understanding the same purchase order, the same inventory position, and the same business rules. And I think it was very useful to bring up the discussion of token cost too. Huge topic right now. Because teams need to experiment alongside ways to assess whether the work completed justify some of the resources that have been consumed. And then measuring that across an entire process gives leaders a much more useful basis for for decisions than just counting generated output alone.

[00:25:40] - [Speaker 0]
Time to think a little bit deeper than that. And a big thank you to Erfan for returning to the show, sharing SAP's perspective on business context, governance, and enterprise AI. You'll find all the resources discussed in the episode notes over at tech talks network dot com. But I'd love to hear how all these questions that we've mentioned today, how are they playing out in your organization? Are data quality and permissions slowing progress, or is measuring value becoming the harder part?

[00:26:12] - [Speaker 0]
As always, thanks for listening to the Tech Talks Daily podcast. I'll be back with another conversation tomorrow, and hopefully, you'll join me again then. Look forward to speaking with you. Bye for now.