Building audit ready AI agents in finance with Avalara

[00:00:06] What happens when an AI agent can calculate tax, move money or trigger a filing, but nobody can explain exactly who owns that decision? And this is a question that matters far beyond finance because autonomy without accountability can turn even the cleverest of pilots into a very expensive lesson.

[00:00:28] And my guest today is Danny Fields, Chief Technology and Customer Operations Officer at Avalara. And he will explain today why many AI agents impress in controlled tests, but struggle with messy production data, old workflows and unclear ownership. So I want to discuss with him today what audit ready AI requires, how CFOs can better judge whether an agent deserves greater freedom

[00:00:57] and why human approval belongs at decisions that are carrying very real business consequences. And the company processes 54 billion tax calculations each year. So Danny's going to be bringing a very useful perspective from a field where accuracy cannot be treated as just a nice bonus. But enough from me. Let's get into it now. Let me introduce you to Danny. So thank you for joining me on the podcast today.

[00:01:25] Can you tell everyone listening a little about who you are and what you do? Hi, Neil. Hi, everybody. It's great to be here. So I'm Danny Fields. I am the Chief Technology and Customer Operations Officer here at Avalara. And if you've never heard of Avalara, well, we are the agentic AI platform for global tax and compliance. And we operate globally all across the world. And we have been building our solution for more than two decades.

[00:01:53] And Avalara delivers what compliance requires most, trusted accuracy. And we help businesses automatically handle the complex, ever-changing rules around taxes, like calculating the right sales tax on a purchase and staying compliant across different states and in more than 190 countries around the world. And we process more than 54 billion tax calculations annually. So my role sits between the technology that we build.

[00:02:22] So I lead the engineering team here at Avalara and how customers actually experience it. So helping customers onboard onto our solutions, helping run some of our managed services. And then if customers have any technical issues with their products, I lead the support organization that helps resolve customers' problems. A little bit about me. I'm originally from Dublin in Ireland.

[00:02:48] And I've been in the United States for the last 32 years, working in different enterprise companies, building cloud applications, working in security. And here at Avalara for the past eight years, building planet scale solutions to process billions and billions of tax calculations and other tax activities.

[00:03:16] And I've got to ask, because I'm a sucker for an origin story. Decades ago, you went from Dublin to the West Coast of the U.S. Is there a story there? And also the tax and technology did something light the spark in you there? Tell me about that story and how you got there too. That's a great question. So it was the work I was doing. I worked for a small little company called Oracle in Dublin. And the work I was doing moved over to the United States back in 1994.

[00:03:44] So I moved with the work over to Oracle's headquarters. Thought I would stay for a year, then three years. And here I am 32 years later. And my family and wife and kids are all here in the U.S. now. So I think I'm here for good. And it's a great question about getting into tax and compliance. I think when the recruiter first called me about the job and told me it's a tax company, my response was click because it doesn't sound very exciting. It doesn't sound very sexy.

[00:04:15] But when I got to understand what we do, what we're doing today at Avalara, when I learned more about the team, about the company and about the problem that we're trying to solve. Today, when we're hiring engineers into Avalara, I'm the one who usually gets on the phone with them after the recruiter to explain the engineering problem we're trying to solve. Like if you think of most of the big cloud companies that you all know of,

[00:04:44] they're all probably solving like one thing. But in the area of maybe it's search, maybe it's a ride sharing app, whatever it is. It's some business application. But in our world of tax and compliance, there are dozens and dozens of different activities that have to be automated. Calculating tax, filing a tax return, handling a notice, processing an exemption certificate and validating it.

[00:05:10] Each one of these activities is an engineering challenge on its own. And the solutions we're building have to work planet scale all across the world in different geographies. They have to be secure. They have to scale to handle billions, many billions of these things. They have to be able to handle tens of thousands of them at the same time. And they have to be super secure.

[00:05:36] So when we explain these engineering challenges to people who want to join Avalara Engineering, it's the same reaction that I had when I joined the company eight years ago. You get goosebumps. It's like, wow, that's a really tough problem to solve. I didn't know tax and compliance was that difficult. And that's why I'm here. That's why I've been here for so long. And that's why engineers love joining Avalara. It's because of the problem we have to solve.

[00:06:06] And now we have AI that can help us solve the problems a little bit easier. But there's still difficult problems to solve. And that's what made me jump to Avalara. Incredible. Incredibly cool story. And thank you so much for sitting down with me today. There's a lot I want to talk with you about. One of the things that set off my tech spidey sensors and wanted to get you on here is I was reading your research, which found that more than nine in 10 finance leaders are currently

[00:06:35] feeling pressure to prove return on investment from AI projects, in particular AI agents, while many are seeing very limited measurable returns. I'd love to start here. I mean, what's going wrong between a successful AI pilot and something that creates that measurable value in finance? You know, Neil, that's a very good question. So a pilot environment, when we're building something in engineering or when a customer

[00:07:03] is running something in a pilot, it's done in a very controlled environment. All of the inputs and outputs are controlled. But in production, things are not controlled. And that's where the big gap lives. So an agent, AI, that worked cleanly in a very narrow test environment, suddenly when it has to deal with real world, messy data, tax data, you can have imperfect inputs. And the workflows don't know how to handle them in some cases.

[00:07:32] So when we surveyed our finance leaders, they said that their AI investments to date have only so far delivered somewhat measurable or limited ROI. So even while 92% of them are under personal career pressure to prove those investments are working, they're not getting the results that they're looking for. The other problem is that we're trying to automate a broken process in some cases.

[00:07:59] We're trying to take a one-for-one process and just make it work faster in AI. And the organizations seeing real returns are the ones who have completely redesigned the workflow around the agent. They haven't just handed the agent an old process or an old workflow and have the agent deal with that. They've completely reimagined or completely re-transformed or transformed rather their process.

[00:08:27] And finance especially, if an output can't be tracked back to a vertical source or rather a verified source or explained to an auditor, it's not just unhelpful, it's a liability to the company. It's such an interesting point you made there. And I'm hearing this more and more around old processes, old workflows that were essentially designed for a completely different era. And it just seemed kind of crazy to be just tagging AI onto those rather than redesigning it.

[00:08:55] Is that message getting through now, do you think? I think it's starting to get through. I think people are starting to realize the entire process needs to be reimagined. And I think that's the key word. It's not just taking AI and sliding it into an existing workflow. It's looking at the entire end-to-end process, the entire end-to-end workflow, reimagining it. And that's the word I would use when you're deploying AI. Don't just deploy an AI agent.

[00:09:25] Consider reimagining the entire process. And there is an interesting tension in the findings elsewhere too. And one that stood out to me was finance leaders are being pushed to deploy agents quickly. It may be unsurprising there. But only 7% say their organization prioritizes governance over speed. So how do you move quickly without creating technical, financial, or regulatory headaches

[00:09:53] that will inevitably surface further on down the road? Anything you've seen there? You know, it's a great question. We understand the pressure. But in a regulated environment, you can't prioritize speed over governance. You actually need a little bit of both. And if you don't do that, you're going to create regulatory risk that will show up at the worst possible moment. So the answer isn't to slow down.

[00:10:18] The answer actually is to make sure you're building governance into your workflows, into your agentic workflows from the very beginning. And you don't want to slow down. So three things you need to make sure that your agent is doing. You need to make sure it's operating within permission boundaries so the agent can only act within your system of record. You need to make sure you've got human approval points at the decisions that carry consequences.

[00:10:44] And you also need to make sure that in your agentic workflow, you have audit trails that capture what the agent did and understand why the agent made a particular decision. So retrofitting governance into a running agent, it's like retrofitting brakes onto a moving car. You can do it, but the results aren't always great. So again, I would say you can go fast.

[00:11:09] We need to make sure governance is built into your agentic workflow from the beginning. And another somewhat worrying stat that you found there was nearly one in four respondents said responsibility for a significant AI agent error would either be unclear or actually sit with nobody. Now, at a time where we're having not just individuals, but teams creating hundreds, if not thousands of agents that have no clear responsibilities is a little worrying.

[00:11:38] So as agents begin taking actions rather than just making recommendations, the question I've got to ask is almost an episode on its own here, but who should ultimately own their decisions? That's an easy one, Neil. So it's the organization that's deploying the agent. They own the outcome, whether it's the finance department or any department in a company, they must own their own outcome. So the accountability doesn't just transfer to the technology team that's implementing the agent.

[00:12:08] The ownership has to stay with the team that owns the business outcome. And right now, most organizations haven't resolved this fully. And you're right. About a quarter of finance leaders we surveyed said accountability for a significant AI agent error would be unclear or sit with no one. And that's real organizational exposure. So the companies doing this well have distributed ownership across finance, compliance, IT and

[00:12:35] legal with documented escalation paths and governance that survives personnel changes. So for example, IT for the integrity of the data, tax and compliance for the actual determination of tax ability and rates and finance for value and risk. So for the finance department, the ultimate accountability for the business value and risk will lie with them.

[00:13:00] And another stat I'll pull out here is that I think it was more than three quarters of respondents like dedicated in-house expertise to fully understand how their AI agents are working. So, I mean, does this create a dangerous dependency on vendors and outside of that, what level of AI understanding does a CFO that could be listening to our conversation today? What kind of understanding do they realistically need? They need a great understanding.

[00:13:27] So the danger isn't just relying on outside vendors who are going to implement a new tool in finance or a new technology, but it's making sure you know what questions to ask them. So if you can't evaluate what a vendor is telling you, then you can't make an informed decision about scope, autonomy or risk. So a CFO doesn't need to understand how a large language model works at a technical level.

[00:13:53] They don't need to understand how a cloud or chat GPT works, but they do need to know certain things. They need to know what data is feeding the agent and is that data verified? They need to know what is the agent authorized to do? And they need to know when does the human take over and when does the human approve? And they need to know when decisions made by the agent, are they documented? And can they be explained later?

[00:14:22] And can they be defended? And that's a governance problem. It's not a computer science problem. And it sits squirrely with the CFO. So the CFO does need to understand the basics about what an agent is doing. And again, they don't need to understand the technology. They just need to understand what the agent is doing. And I don't want to be all doom and gloom here, because I think if we go back, what, 18 months ago, nobody was talking about agents.

[00:14:48] So I think it's important to recognize that we are very, very early on on this journey. But less than half of finance leaders are highly confident that they could actually explain an agent's actions to an auditor or a regulator. So what does an audit-ready AI agent, what does it actually require in terms of explainability, records, controls, and evidence? So it's not just a black box that nobody really knows what goes on there. Yeah, you know, that's a great question.

[00:15:16] So there are probably four things that the agent requires. So first of all, a complete audit trail. Everything the agent does, every action, every input, every output, everything has to be traceable to a specific rule or to a specific data source. So that's the first thing. The second thing is explainability at the decision level. So if an auditor comes in and takes a look at what the agent has done,

[00:15:41] the agent doesn't need to take your word for why the agent did a particular treatment or took a particular action. The auditor needs to be able to look at the audit trail and everything needs to be explainable. So that's the second thing. The third thing is a documented human approval system needs to be in place. So the record will show to the auditor which actions required sign-off by a human,

[00:16:10] who gave the approval, and what actions were just taken by the agent on their own without a human. And then the last thing is tested incident response, because nearly half of finance leaders who responded to air survey have plans that are either untested or still in development. So the four things, again, would be an audit trail, explainability at the decision level, documented human approvals thresholds, and a tested incident response.

[00:16:40] And that very much ties in with what many of the respondents told you there. They want agents grounded in verified financial tax and compliance data and operating with existing systems of record. Makes perfect sense. But how important is the quality and authority of that underlying data when determining just how much autonomy an agent should receive when you almost hand over the keys to them? You know, that's the key.

[00:17:07] Data quality is foundational behind every agent and behind every agentic workflow. And it's the variable that should set the dial on autonomy. When we asked finance leaders what would most increase their confidence in expanding AI agents in their departments, their top answers were, well, the agents operating within existing systems of record, outputs grounded in verified tax and compliance data,

[00:17:35] and validation against known compliance requirements. That's a data trust requirement. So they've got to trust. You've got to trust the data. And an agent's autonomy should be proportional to the verifiability and currency of the data it's acting on. And the other thing that you need to take into account is that tax law changes constantly. And we see that inside of Avalara daily.

[00:18:00] An agent has to be grounded in tax content and tax law that's changing daily and sometimes multiple times a day. So you have to make sure that your agent is up to date on the latest tax rates, latest tax rules, latest tax laws. And if the agent is not, then you're just creating a liability. So again, data is super important. It's foundational to everything that an AI agent does

[00:18:30] and that an agentic workflow is built upon. And I also wanted to highlight today that you guys process more than 54 billion transactions annually. So I think you have quite a unique vantage point on where automation meets regulation at an enormous scale. So from what you're seeing and hearing here, which tax and compliance activities are becoming more suitable for agentic automation?

[00:18:56] And where should human judgment remain firmly in the workflow? We hear a lot around human in the loop, a bit of a buzzword over the last few years. But where should that be? You know, you're right. We process about 54 billion tax transactions annually. And we know probably better than most where automation and autonomy plugs in and where it doesn't into a customer's workflow.

[00:19:23] So when it comes to tax calculation, that's what we would call something that is deterministic. And we've built an engine that can process a high volume of tax calculations following rule-bound processes with a clear right answer. When you calculate tax, the answer has to be right 100% of the time. So things like calculating tax, preparing and filing a tax return, processing an exemption certificate,

[00:19:53] doing a HS code classification, taking a product and classifying it to the right HS code, monitoring your environment for regulatory tax changes. All of these activities are deterministic. There's a right answer. And using an engine in a deterministic way to process these activities at high volume, at high speed, in a very secure way,

[00:20:20] that's where you can use agents and agentic workflows. Where humans are needed. Humans are needed where stakes are high and where some interpretation is required, where there's genuine ambiguity. So scenarios maybe around authorizing certain payments, maybe responding to a certain audit notice or other notice that comes from the tax jurisdiction,

[00:20:49] maybe a notice that is tied to certain business implications where a human has to provide certain judgment. So the principle we follow is let the agents handle scale and let the humans focus on judgment for decisions that carry real business consequences. Do you need AI agents that you can trust? Well, with an AI data layer providing real-time connection within your data platforms,

[00:21:19] you can trust your agents to provide accurate solutions. So scale your business by trusting your agentic AI accurately getting the work done for you. Trust its capabilities with Denodo. And you can do that by simply visiting denodo.com to learn more. At the end of every episode, I try and give people listening some valuable takeaways. So if we do have a CFO listening today who already might have several AI pilots underway,

[00:21:47] what kind of practical tests would you recommend before giving an agent greater autonomy? What should they be able to prove around ROI, accuracy, accountability, security and governance before finally moving into production and moving into that confidently without any fears of they've missed anything? Great question. So for ROI, the first one, so can you quantify the value being created?

[00:22:13] It takes a lot of time and effort to go build and deploy an agent or completely replace a workflow with an agentic workflow. So you want to make sure you're delivering a good ROI. So it's not just time saved. You want to make sure you're improving accuracy, you're reducing risk, and you're improving your audit defense capacity. And if you can't measure it, then you can't defend against the expansion of building and deploying these agents.

[00:22:42] Accuracy. So has the agent performed at the level you'd expect from your best team member? And is it accurate over a sustained period of time? Does it get the answer right all the time and across the full range of inputs that the agent will face in production? But earlier we said that agents behave a certain way in pilot scenarios.

[00:23:03] But when you deploy the agent or the thing into production, will it always work well with different kinds of messy data going into the inputs? Accountability. So if this agent makes a significant error tomorrow, who owns the mistake? What's the escalation path? And who's going to fix the problem? What's the remediation process? And if the answer is, well, we'll figure it out when we hit a problem, well, then your agent is not production ready.

[00:23:33] And security. So if it's the agent, what data does the agent have access to? So do you have guardrails in place? Does the agent only have certain permissions to access certain data? Can the agent only act within certain boundaries that you've set? Or does the agent have access to all data? Well, if the agent has access to all data in all of your systems,

[00:24:00] then maybe it's not yet ready to be rolled out into production. And then finally, governance. So does your internal control framework actually reflect that you now have agents in the loop, in addition to humans, that are now taking actions or recommending actions? And when we did our survey, 30% of respondents to our survey, they hadn't updated their controls in the past 12 months. So you're right here.

[00:24:29] You've listed the right areas that we should be looking at before we deploy an agent, Neil. ROI, accuracy, accountability, security, and governance. And as we mentioned earlier, you guys process more than 54 billion transactions annually and support 200,000 plus direct and indirect customers globally. And throughout your career, you must have seen so many big changes. And I mean, we talk a lot around the changes in the last two years and five years.

[00:24:59] But when you look back at your entire career, what excites you about where we are now and where you're heading now as a company too? I think it's because we're able to go so fast. I mean, I'm old enough to have experienced the internet. I mean, how did we do anything before the internet? Or how did we do anything before we all had mobile phones or portable computers in our pocket? And then we went to the cloud.

[00:25:26] And here we are today with AI. And we've had AI for years and machine learning for years. But generative AI over the past two years have made my industry move so fast. So inside of Avalara, we're building systems and applications to have to run a planet scale to automate what our customers are doing in all areas of tax and compliance. And today, I mean, we're just going so fast.

[00:25:56] I mean, we're going nearly 10 times faster today in our engineering team than we did last year. So AI has made us go faster. It's given us the ability to automate things that were so difficult to automate in the past. We're building new experiences for customers that are much more intuitive and easier to use. Right in the old days, a couple of years ago, a year ago, today you go to applications and you spend all of your time clicking through business applications.

[00:26:26] And today with generative UX, you can have a much more intuitive experience for a customer that makes business applications so much easier to use. So everything is changing in the cloud. Everything is changing in business applications. Everything is changing in engineering. And we're delivering solutions to our customers that are much, much better than they were even six months ago. So my world is turned upside down as we go faster.

[00:26:55] And I think it's all about speed and velocity. We're able to do things in a much faster way today than we could six months ago, 12 months ago. Yeah, I completely agree with you. It's something I'm seeing across every industry. And I think it's something that we're all on together and love what you're doing here. And for anyone listening, I will include a link to the report that we've referenced today. But anywhere else for people listening, if they want to connect with you, with your team,

[00:27:22] and keep up to speed with all the latest announcements coming out throughout the year? Yeah, probably two places. You can find out more about me and about Avalara at avalara.com. As we launch new products and new features, you'll see information published there. And you can connect to me directly through LinkedIn. So go look me up on LinkedIn, Danny Fields, working at Avalara. And send me a connection request on LinkedIn.

[00:27:48] And I would love to connect with you and love to talk to any of you even more. Well, thank you so much for sitting down with me today. We covered a lot in a short amount of time today. I think there'll be a lot of words of comfort as well. And also actionable takeaways, what people should be looking out for, how they can improve. And take away some of that fear. But I will include links to everything that you mentioned. I encourage people to reach out, connect, stay up to speed with everything that you're doing.

[00:28:17] Again, just thank you for your time today. Really appreciate it. Thanks, Neil. Great to be here. I think Danny left us with some great advice there. Let agents handle volume and let people own judgment when the consequences really matter. Before any finance team moves an AI agent into production, it should answer five questions. What value does it create? Does it stay accurate with messy inputs? Who owns the error?

[00:28:46] And what data and permissions can it access? And can every decision be explained later? If any of those answers are vague, the pilot probably needs a little bit more work. But I also love Danny's reminder that governance belongs inside the workflow right from day one. Adding it later is, in his words, like fitting brakes to a moving car. Which sounds exciting, only if you're nowhere near the car. So a big thank you to Danny for joining me today.

[00:29:14] I'd love to hear more from you. How much autonomy has your organization earned the right to give its AI agents? Let me know. TechTalksNetwork.com But that is it for today. I'll be back again real soon with another guest. But thanks for listening as always. Bye for now.