Keeping the Human Voice in AI Translation With DeepL

[00:00:00] Your agentic AI might not be secure even with real-time data and proper guardrails. But Denodo makes sure your business has every avenue covered. By placing all your data platforms under one AI data layer, your business can reach semantic consistency safely and securely. So get your agents on the same page by visiting denodo.com and you can learn more about how to

[00:00:29] start trusting your agents to make business decisions. But now, back to today's guest. What does a business lose when the person with the best answer cannot express it confidently in the language that everybody else is using? This is something that I've seen time and time again in corporate settings, especially between those meetings with stakeholders and techies.

[00:00:57] Well, my guest today is from a company called DeepL and together we're going to discuss what real-time voice translation could change for global teams. We talk about the details that make a conversation more human, from hesitation and emphasis to recognising who is speaking. And getting the words right, yeah, that is one challenge. But keeping the conversation

[00:01:21] flowing without losing the speaker's meaning creates many more. So my guest today will explain how DeepL approaches these trade-offs, why unfamiliar acronyms can still cause problems. And anyone that sat down with an IT team will know all about that. Something we try and avoid on this podcast where possible. Also, what buyers should ask about data handling before bringing translation

[00:01:47] into those everyday meetings with stakeholders. And I'll also ask the question any podcast host would have always eventually ask, could this help the show reach listeners in other languages? Imagine that, a German, French, Japanese, Asian, Neil. There's plenty here though for anyone who's worked crosses borders, teams or time zones. Love to hear your thoughts on this one, but enough from me. Let me introduce you to my guest now.

[00:02:15] So thank you for joining me on the show today. Can you tell everyone listening a little about who you are and what you do? Yes. Hi, Neil. Thank you for having us and having me. I'm Gonzalo Gueyeola, also known as Gigi. I am the Chief Product Officer at DeepL, who is the world's best language AI company. So all the way from pure text translation to document, all kinds of media. And most recently, something we're very excited about, voice-to-voice, live human-to-human translation.

[00:02:45] Awesome. And there's so much I want to talk with you about this today, because I think when global teams struggle to work across languages, I'm curious, where does the biggest business cost usually appear? Is it slower decisions, weaker collaboration, lost context, or is it something else? What do you see here when you're having these conversations? Oh, that's a great question, Neil. Right at the ROI of all the work we do. I think you mentioned

[00:03:12] a few that we see that are fairly obvious. We've had examples of customers telling us, hey, you know, once we deployed voice, we were able to immediately reduce the time it took to make a critical call, to get better nuance. I think one of my favorite ones that is least obvious, that is less obvious when you first start out, is the cost of diversity and inclusion. Because a lot

[00:03:37] of people that have not been trained or are not as proficient in, you know, the lingua franca of business might be English in most cases, they still have tremendous knowledge of their local market, their product, their experience. And so they can really contribute in the organization or in society. So that cost for us, it's hard to put a euro or a pound or a dollar figure to it, but it is very, very real. On top of everything else that you have seen, the delays that add up, sometimes the wrong

[00:04:07] decisions because you've missed a word or context or you just couldn't be able to get across. So it is surprising once you start to look into it, how costly the language barriers across all these different modalities really are. And of course, real-time translation isn't simply about just converting words from one language into another. It's much more complicated than that because people interrupt,

[00:04:35] they hesitate, they pause, they emphasize certain words, change direction, and maybe ask questions in a variety of different ways. I know one of the big reasons that voice commerce has not took off particularly well is because, I don't know, if you bring up a pizza place, we all will ask for a pizza in a multitude of different ways. What is it that makes preserving all that in a live conversation so

[00:04:59] technically difficult? That's a great question. And yes, it is. I think this is part of the reason why we have, since the early days of science fiction, always been in love with the idea of the universal translator and still, you know, these decades later, we're getting close, but we're still not there yet. Yes, you, you, there are environmental factors, there are language specific factors. So how language

[00:05:24] gets constructed, how long do you have to wait until you have enough meaning and context? Certainly, more interestingly, even not just the translated words, but the translated tone and the human intent behind it. How do you get that across? All these things individually are hard to do in AI models. When you put them all combined with a latency that needs to feel natural, because speaking to each

[00:05:50] other is perhaps one of the most natural things that we have learned to do through millions of years of evolution. So if that doesn't happen, if it's not fast, precise, then you just lose track, it gets in the way. And our goal is to not get in the way. So it's a series of on their own, technically challenging constructs and capabilities. But when you string them together, it really makes for a difficult challenge. And that's why we've been collectively as an industry

[00:06:17] trying to solve this problem for the last few decades, but we're getting very close. And one of the things that stands out about DeepL Voice is you can now retain so much more of a speaker's distinct vocal characteristic across languages. So urgency, hesitation, emphasis, and even the difference between a statement and a question can carry through just by that intonation there. So why does preserving how somebody speaks matter almost as

[00:06:45] much as that translation process and what they say? Again, it has to do with the natural evolution of how we've learned to communicate with each other. A significantly higher percentage, studies vary here, but a significantly higher percentage of what you're trying to get across in a conversation comes not from what you say, but how you say it. So all the things you've mentioned, the urgency, the tone, the almost cultural nuance of what it is that you're saying,

[00:07:15] even the matching my voice in a meaningful way so that we don't all, in a meeting of 10 people, we don't all sort of sound the same. That's the old robotic way that we've learned to identify as poor quality AI. All those things are maybe the final frontier about people being able to really communicate in multilingual environments and just becoming part of the ambient. So we recently

[00:07:41] launched a series of new model and product capabilities that really target that specific piece, not just the accuracy and the fluency of what was said, but to actually transfer over the intent or the way in which you have done it so that it sounds more natural. And across language barriers and sometimes across cultures, you can really understand the multitude of aspects of what

[00:08:06] was trying to be said. So it's clearly the next frontier in these voice models. I think once we do crack it at scale, which we are again in the process of doing, it will open up a number of use cases across our personal lives, across our business lives that we feel very excited about. And you mentioned that you don't want everyone sounding the same in a meeting. And I would imagine it becomes especially important in meetings with multiple participants, because if everyone emerged

[00:08:33] from a translation sounding pretty much the same, we'd potentially lose a lot of those human dynamics of the conversation. So how is it you preserve individual speakers so colleagues can continue to recognize who is talking naturally across languages? Well, yes, absolutely. I mean, if you could imagine that scenario of like 10 people in a meeting and it's always the same language or the same pace, the same tempo, the same type of just very rhythmic

[00:09:02] and always the same discussion, that would not be a great experience. And you would eventually tune out or it would be distracting. So what we try to do is we certainly isolate out the different speakers in the conversation. And through AI technology, we try to match as many of the non-language specific elements of their

[00:09:25] speech as possible, you know, their tone and their type of voice and the rhythm so that when you hear it on the other side, you can absolutely tell who that person was, where were they animated or not? Were they excited? Were they concerned about something else? And that doesn't again, doesn't get in the way. So there's a lot of like highly convoluted and technical ways we use behind the scenes to be able to do that. But the net,

[00:09:51] the outcome feels very natural. It feels essentially you're still hearing a person in the original language. So there's always a little bit of that. We just duck that sound a little bit more. And then what we do is we translate over, but you can absolutely tell it's the same person doing it. You can absolutely understand the emotion and the intent. It's very natural behind it. And one of the things that I think is incredibly exciting and what you've achieved here is you've now

[00:10:16] reached low latency voice-to-voice translation across more than 30 different languages. So I've got to ask, how close are we to the point where two people who don't share a language can simply have a natural conversation without even consciously thinking about the technology that's translating between them? It feels like a really exciting moment, but how close are we to getting that part? We are very close. And I think our experience and our research tells us that once you get used to

[00:10:44] that initial slight difference of still hearing the person and there's still a little measured in milliseconds delay. But once you get past that, it's no longer a barrier. So we have literally millions of minutes of translation happening every week on our platform. And you can tell that the more people use it, the more they achieve exactly the experience that you're talking about. This feels just very natural. You don't add delays. You don't have confusion. There's no round trips.

[00:11:13] So I'd say at the global scale on somebody, it doesn't matter if they're in a kindergarten. This is one of the cases we have right now where parents and teachers talk across the world. They cross that language barrier when they're talking about their kids or in a high stakes merger and acquisition meeting. We're getting to the point where after a very tiny adaptation period, the discussion is very natural

[00:11:41] between two people or a set of people and the language no longer becomes a barrier. It actually becomes almost a point of connection between folks because they can hear themselves in the language that they understand. So empathy actually goes up. And I'm curious when it translates a speaker there, is that does it retain the speaker's actual voice or is it a different kind of voice?

[00:12:07] It does. Of course, we give our users and our customers options. Some people want to have the capability because they want to use more of their vocal fingerprints and match this, what the speaker is saying. That is absolutely possible today. Of course, that is also up to control, to the folks utilizing. Some folks don't feel comfortable with that. That's fine as well. But yes,

[00:12:31] we currently have capabilities that allow you to retain and maintain a lot of the speaker identity, including their voice, again, their tone and everything else around it. So from a selfish point of view, Tech Talks Daily could go out in 30 languages within a couple of years. Oh, 100%. 100%. Yeah, we already have a lot of folks experimenting with... Of course, we try to specialize in a lot of things that are real-time, but you could use the same technology fairly

[00:13:01] quickly to also do more, let's say, at scale performance. And of course, media companies and podcasts are a great use case. And DeepL Voice is now generally available for online meetings, following the earlier rollout for in-person conversation and of course, DeepL Voice API. And when I was doing a bit of research on you, I was reading you've also introduced a desktop app that works with Zoom, with Microsoft Teams,

[00:13:27] with Google Meet, all the big places that we all communicate on without requiring separate platform integration. So why was creating that one seamless experience across three very different ways of communicating? Why was that so important? And what does that change for enterprise adoption, would you say? So that was a very conscious decision we made because when we first started, to your point, we started rolling this out in our mobile app that's used by millions

[00:13:55] of people. And that was a more controlled environment, let's say, one-on-one discussions, could be the kindergarten scenario we talked about, could be somebody traveling or a frontline worker, a driver for a company. But when we started going into the online and hybrid meeting type scenario, what we realized is that the enterprise environments and folks talking from one company to another,

[00:14:19] the ecosystem of tooling that was used, you mentioned Zoom, Teams, Meet, it's so varied that we wanted to give people a home. And we understood that this became so critical to the workflows of global work, to the workloads of global workforces, that we really wanted to give them a single, well-managed experience that worked across all of these platforms. There were always nuances in

[00:14:45] these platforms and it worked slightly different. And you know, you would have to then make compromises and the smallest possible, you know, and basically use the lowest possible denominator on the technology. We didn't want to do that. The problem was so important that we decided we needed to roll out our own app that would then allow an enterprise user to operate across all of the myriad of complex ecosystems they have internally, but also when they speak to their customers and their partners.

[00:15:12] So yeah, the DeepL voice app gives those businesses that very consistent experience. And because frankly speaking for these enterprises, that ability to integrate and work across this disparate set of tooling is just as important as translation quality and tone preservation. And DeepL has to balance translation accuracy, speed, conversational context, and now the

[00:15:40] characteristics of a speaker's voice. It's absolutely phenomenal what you've achieved with technology here. But when some of these requirements, do they inevitably compete with each other? And how do you decide what can be compromised and what absolutely cannot? I would imagine it's quite a balancing act. There's been a few heated conversations, but how do you get that balance right? Right. We think of this almost as a pyramid and a model of needs and wants from customers and users.

[00:16:07] And at the very base of that model is accuracy, trust, security, which we know is absolutely fundamental. So DeepL historically was born out of Germany. So trust, privacy, security is very much paramount to everything that we do. And that is like the baseline, right? So you do not get to sit at the table with your users and your customers if you do not get accuracy, trust, security right. And then on top of that, to your point, it is about playing around with different scenarios around accuracy, speed,

[00:16:36] latency, and trying to create, to give our research and engineering teams enough leeway so that they can optimize for different things in different scenarios. So it's an ongoing, of course, an ongoing challenge. The recent launch we've done was done after we have conducted extensive testing with a number of customers where we think we've really nailed the point of that latency. So how long you have to wait for the translation to feel real time without sacrificing that accuracy and

[00:17:05] And we're still giving folks the required transfer of all the non-verbal, if you will, or the non-translated parts of the communication. So it's a complicated topic. And to your point, sometimes we get into heated discussions, you know, which direction do you go? But the most important thing is, it is always based on the deeper history of accuracy, trust, and certainly governance across the enterprise.

[00:17:31] And when it comes to working with AI, I suspect we've all seen or heard examples around AI hallucinations and that spotting of patterns and just telling us what we want to hear. So how do you test whether a translated conversation has really preserved that original intent and meaning rather than simply producing words that sound correct? And also, how are you preserving elements such as the speaker characteristics? Does measuring the quality of translation itself need to evolve?

[00:18:01] I'm conscious of throwing about three questions in one at you now, but tell me more about that and how you get around that. Well, this for us is an evolution of also capabilities that we have built throughout the years because VPAL has always had this issue of how do we ensure human-grade quality, not the general metrics and benchmarks. We know AI companies like to benchmark max. We try to really do our benchmarks are what is useful in production for customers.

[00:18:28] And we employ a number of internal techniques to test with real users and to test with a wide variety of users to support that spoken communication. So I think we had a bit of a head start because of all the work we've done with millions of end users and in our core translate product around text and around documentation. And we have applied many of those same techniques to voice. Now, what we've also found is you can get pretty much like 90% there. And then there is some work where the models, it's not technically a hallucination.

[00:18:58] They don't make things up, but they might misinterpret. But if you are, for example, in an engineering department in a manufacturing company, the amount of acronyms and things that they use, like words that are weird, that just play out in their context is so big that sometimes they trip up the models and the translations become funny. So what we have is customization capabilities so that we capture your glossaries and your style rules. How do you want your brand to be reflecting on how you speak and how you get translated?

[00:19:27] But also spoken terms, you know, how do you say acronyms and things like that? So the system is also able to learn or you can teach the system so that over time users approaches really something that is very meaningful for the context of where the product is being used. And I also wanted to highlight that this is not a new solution. You've been working on this before AI became cool and everyone was getting obsessed about it.

[00:19:56] I think I had Steve Rotter. He was your CMO. I'm not sure if he's still there. I spoke to him, what, three years ago when he was telling me all about the technology and stuff. So again, for people listening, what should enterprise buyers be asking about security, data handling and governance before introducing real-time voice translation into their everyday meetings? Because we've been on quite a journey over the last few years. How do you see that security element there? It's probably a question you get asked a lot, right? We get asked that question all the time, right? Neil, I think you're absolutely right.

[00:20:26] I mean, if you think about the use cases in which real-time human-to-human speech translation can be used, it is hard to think about something that can have either more proprietary or sensitive data, right? So for us, I think we always tell customers, how is it that we don't process their data, don't use their data to train our own models, how the technical architecture behind the scenes works, what kind of compliance at the stations we have.

[00:20:54] All of these things are fairly standard. And good enterprise AI governance should still apply. You should still think through how serious is this for your vendor. And I'm happy to report, again, like I said, DeepL, there's very few things that are more critical at the DNA and core of DeepL than privacy and security for our users.

[00:21:20] Just by virtue of where we were born and what kind of reputation we've built. So, yes, I would highly recommend that buyers take a strong, strong look into how their data is processed from the technical architecture to the policy layer within all of their vendors. And, of course, we are only literally a couple of months or even maybe weeks now away from life in 2027.

[00:21:46] If we look into the future, if that ultimate ambition is for multilingual conversations to feel like any other conversation, what still needs to happen before that technology or before the technology really becomes invisible, fades into the background? How long are we from that moment and what needs to happen, do you think? It's somewhat hard to predict the future.

[00:22:10] But I would say across all different modalities, and this could be video, text, document, voice, I think we are in the 50% to 80% range of being ambient AI. Like just fades into the background, just work. You don't even think about it. Voice might be the final frontier just because of the very visceral nature. It's very interesting to watch people use our product for the first time. I always love to go on customer calls. You show it, you turn on the product, and their first reaction is like, ah.

[00:22:40] It's just like kids on holidays. And I think we're, so that moment's already there. That moment of wow is already there. Now, for it to be truly embedded and augmented, we still have continued work to be done on the technical architecture, on the AI models. But I would estimate by, if we talk in one year's time, like you did with Steve, at that point in time, I would estimate that enterprise adoption has skyrocketed.

[00:23:09] And that we will probably not just be fully natural, but already quite embedded in critical workloads like contact centers or safety briefings or M&A transactions and all of those things. This will be just a new natural. I think we're in 12 to 18 months distance from that. Wow. Wow. Exciting times ahead. Sounds like we are going to have to get you back on next year. Yes. How's things going to be progressing?

[00:23:36] I'm curious, from the conversation you're having, what kind of feedback are you getting here? And are you hearing about more and more use cases that maybe you guys have not even thought about? That is exactly what is happening. So we thought we knew all people were going to use the product. And then, of course, no plan survives contact with actual customers. And customers are using us a lot in contact center solutions, a lot of it. And we have customers that use us in parallel with human interpreting so that they get both signals at the same time.

[00:24:06] We have a lot of folks trying to do hybrids and massive events. So, for example, we were just at Dreamforce, Salesforce. We just did all of the translation for Dreamforce. And that is magical to just watch it happen in real time in the show floor. So we are getting pulled in all of these different directions. It's an interesting strategy problem. We're getting pulled into all these different directions because there is a latent demand in the industry.

[00:24:32] Once people get to a point where they believe the use case is going to be technically feasible, they will pull you into all these different directions. So it's a very magical time to be developing such a technology. Incredibly cool. Finally, if anyone listening wants to find out more information about anything we talked about today or even have a play with the technology, where should they go? Where can they find more information? So for sure, go to deepl.com. In the homepage directly, you can just try voice.

[00:25:01] And all of our other translation products, they're pretty amazing if you ask me. Of course, follow us on our socials as well. We keep posting updates specifically on voice. And that is it. Well, I've loved chatting with you today, exploring from the beginning the business problem that we're looking at solving here, right through to the technical challenges. The customer feedback, how it almost develops a life on its own. I'll include links to everything that you've mentioned there. Maybe find a few videos I can embed as well just to further bring that to life.

[00:25:30] Thank you for thinking, Bigo, and for sharing your story today. Really appreciate it. Thank you, Neil. It was a pleasure. What stayed with me after our conversation today is that the knowledge that can remain unheard when someone is working in a language that they don't actually speak confidently. Better translation could give that person a stronger voice. But choosing the technology still requires very careful questions.

[00:25:55] Does it actually preserve intent when colleagues interrupt or use unfamiliar terms or acronyms? Can people recognise who is speaking? And what happens to the conversation data? And what control does each speaker have over the use of their voice? And I'd begin with a familiar meeting. Test the language, the terminology. All the things that matter to the team. And then ask participants whether they could contribute more comfortably.

[00:26:22] And speed and accuracy, yet they deserve attention alongside that human experience. But a big thank you to my guest for joining me today. Remember, you can find out more about DeepL's products at deepl.com. But as always, love to hear your thoughts. Where would a better conversation across languages make the biggest difference in your working day? And how would you judge that result? As always, techtalksnetwork.com. Love to hear from you on this. But it's time for me to go, I'm afraid.

[00:26:51] I'll be back again real soon with another guest. But let me know your thoughts on this one. And I'll be back with another real soon. Thanks again. Bye for now. Bye for now. Bye for now. Bye for now. Bye for now.