Finding Potholes Before They Form With AI and Univrses

[00:00:03] Could the solution to all the potholes in your street and the road you travel on to get to the office, could the solution be already driving past your front door every week? Well today I'm going to be speaking with the CEO of the Stockholm-based Universes. They're doing something incredibly cool by turning waste collection trucks, taxis and passenger cars into AI-powered road monitoring agents.

[00:00:30] So I want to learn how they're attaching cameras that can detect surface damage, obscure signs and failing road markings. And how it's helping repairs cost far less because authorities are acting so much earlier. My guest has his own inspiring origin story too, which runs from engineering Formula One cars at Red Bull Racing to autonomous aircraft and computer vision for road infrastructure.

[00:01:00] I think he brings quite a rare perspective on how sensor data actually can drive useful action. And we will also compare Waymo and Wave, examine Universes' work with Pirelli and think about how cities can collect valuable infrastructure data without turning every vehicle into a rolling privacy headache. If you work in this industry, prepare to be amazed.

[00:01:26] It certainly set off a few light bulb moments in me, but I'll be interested in your thoughts. But right now, let me introduce you to my guest. So thank you for joining me on the show today. Can you tell everyone listening a little about who you are and what you do? Yeah, sure. My name's Jonathan Selby. I'm the CEO at a Swedish company called Universes. There's about 50 of us based here in Stockholm. The company's expertise is computer vision.

[00:01:53] So we're really good at converting images and videos into something useful, into useful data. And so we do that in two ways, partly with automotive, helping cars drive themselves, but also now increasingly using the data from those cars to help manage the road environment, to help find where potholes are, find where traffic signs are broken, and help the road environment be safer for us all. Well, there's so much I want to talk with you about today.

[00:02:20] But before we talk about what you're doing now and the incredibly cool technology that you're working with, I always like to dig a little bit into my guest's origin story. What put them on this path? What lit the spark? Now, a quick Google later, I learned that you went from Formula One engineering with Red Bull Racing into autonomous aviation, now AI-powered road infrastructure. So tell me more about that origin story and what all these worlds taught you about sensing. For sure, yeah.

[00:02:48] So I originally had quite a technical degree, applied mathematics at Bristol University, and I wanted to use that learning in my career going forward. But I didn't want to do something that was – I wanted an interesting application, something that was quite fast-paced, dynamic, and got involved in motorsports. And that's how I ended up at Red Bull Racing. And so my role there was to try and engineer the car to go faster, basically. So I worked a lot on the gearbox, worked a lot on the aerodynamics.

[00:03:15] It's fantastic exposition into the world of engineering and trying to engineer things to be to the absolute optimum, basically. And key to doing that is harnessing the data coming from the car. So that was really where we got involved in data analysis, data mining, to find insights and ideas for how we could do that. But I began to find that that was quite specific and detailed.

[00:03:42] It was Formula One's an exciting and dynamic environment, but it's quite rarefied and quite focused. It's very narrow, if you like. So I ended up trying to sort of broaden my horizons a little bit. And that was when I got involved with a company developing unmanned systems, unmanned aerial vehicles. So, again, trying to develop technology, this time to operate on its own in the real world.

[00:04:05] And, again, you need to, in order to do that successfully, you need to understand what the system is doing as it operates in the world. And the only way to do that is through data from sensors. So a lot of time working with the drones, the UAVs, a lot of crashes, unfortunately, but that's the best way to learn, I suppose. But we managed to develop some great systems, mainly focused on surveillance applications. I worked with the US DOD, UK MOD and other organizations like that.

[00:04:35] And really, that was really the transition to computer vision came because these drones started to use cameras a lot more over the last 10 years because of the advent in some of the latest AI technologies. So I was working at the cutting edge, helping to use cameras, not just to look at targets and get insight to what was happening, but also to use the camera feed to help navigate. So the drone itself knew where it was, knew how to navigate in the world, perhaps when you didn't have signals like GPS.

[00:05:05] The NSS was denied, but from the drone, so it could still navigate with the camera on board. So then made the transition to Sweden and got involved with universes. And that was where I sort of found this new exciting applications in automotive initially. So working with some major global OEMs, processing the data from cameras on board the cars, helping them to understand where the obstacles are, what not to hit, more or less,

[00:05:33] and then making the transition to leveraging that data for the road environment. And just go back to your F1 days just for a moment. I spoke with Ruth Bushcon earlier this year, and she was talking about how engineers are using technology at F1 to read a million data points every second. It's phenomenal, the kind of information that's being monitored in that sport, isn't it? For sure. And I think the quantity is one thing, but actually the really, with the latest and greatest, is how you get the insight from all of that data.

[00:06:03] How do you filter and sift it to find the nuggets of gold that says, this is what's happening, this is where it's going wrong, this is where there's opportunity to improve and change. And I think that's where perhaps, you know, as we're going to talk about, the AI has really given these engineers a step change in ability to do that, because it's one thing to get all that data in the first place, quite another to interpret it and derive meaning from it. Fast forward to present day and the work you're doing now.

[00:06:31] For anyone picturing, I don't know, an expensive new sport city infrastructure, your approach is very different, because you can turn vehicles already driving around a city, including everything from bin lorries and taxis, into almost road monitoring agents. So tell me about that. How does it work? And what can these vehicles actually detect? Yeah. So it's what we've, in working with autonomous cars,

[00:06:57] we ended up developing algorithms that process the images from video cameras onboard these vehicles. But what we realized, we could work with cameras that were embedded at the manufacture time, but we could also retrofit. We could also take cameras that we had bought off the shelf and fitted with processors and retrofit them to any vehicle and run the same algorithms. This time, we're not trying to make anything autonomous. We're just trying to mobilize them, turn them into these data harvesters, as you described them. So we've had a camera looking forward.

[00:07:27] It looks a bit like a dash cam. And so the camera is looking at the scene that you can see as a driver. And we process the images from that video stream, and we're able to extract information about the world. So you mentioned bin lorries. We work a lot with those. So as fleets of bin lorries cover most of the cities in Sweden, all of the street network in two weeks, every two weeks.

[00:07:52] So we're able to collect a kind of digital representation of the city every two weeks. And when you speak to most of the cities before we met them, that's the kind of data they would collect every five years. So it's now a sort of step change, if you like, in update rates of what's happening. And of course, what these cities don't want is lots of video, lots of data, as we described before. What they want is the insight. They want the nuggets of gold. And so what we do is process those video streams actually on the vehicle itself. We don't stream video all over the place.

[00:08:22] That helps with data privacy. It also helps with costs. It helps with speed of iteration of results, if you like, to be able to get the result very quickly. So we process those video streams. We extract useful information, like where there's surface irregularities that need to be repaired, like potholes, where there's vegetation growing across traffic signs, where there's roadworks, which is often quite difficult to track. So all sorts of information about the road environment that we then serve up on a map.

[00:08:50] We also give them insights and recommendations about what actions they can take. And so suddenly, having had a road infrastructure and road network that they didn't know anything about for every five years, suddenly they have an update every two weeks with a whole list of actions they can take, costed, appropriately prioritized, that allows them to intervene, make better decisions, better use of the taxpayer money that they have available to them to make this integrate. Wow, it is absolute genius.

[00:09:20] And for anybody listening here around the world, in the UK, there is a huge problem with potholes, a big hot topic. No matter where in the country you live, there will be a lot of problems around that. And just to bring that topic to life, if we zoom in on that, I was reading that one Swedish council reportedly reduced potholes from around 3,000 to just 900 in six months. Well, road condition updates that previously happened every five years now happen every two weeks.

[00:09:48] What changed operationally when authorities suddenly had that level of visibility at their disposal? Well, I think the interesting thing is that the data is one thing, but you have to really tie it to a use case, a value that's going to really make a difference in the context of the city operations. And what we were able to recognize is that the cost of intervening and repairing these potholes is so much cheaper when the pothole is at an early stage of its life, if I can put it that way.

[00:10:17] If it hasn't evolved into a major, major problem, basically. And so the challenge is to detect all of these issues before they become a big problem. And that's where these update rates every two weeks as opposed to every five years become really valuable because you're able to identify and target the problem before they become very large and very expensive. And what that means is when you intervene, the cost is up to 15 times cheaper when you intervene at an early stage

[00:10:46] than when you've waited for it to get to be a major issue that's more expensive and also a larger safety case. And so what changed was that it started to intervene, thanks to the data that we were providing at an earlier stage. It's more cost effective and they stemmed the tide of production of the pothole, if I can put it that way. They somehow reduced the rate because they were intervening at a much earlier stage of the development of it.

[00:11:11] And I think fixing a pothole is one thing, like you say, but being able to predict where one is beginning to form, that sounds much more interesting and proactive. So what signals can AI detect before humans would recognise a problem and citizens going out there, we take measures and things. If we get to it before then, how does preventative maintenance maybe change the economics of looking after our roads?

[00:11:35] Well, I think the camera is able to see defects on the road that you and I might say, maybe that's not yet a problem. Perhaps a road expert would, they would be able to identify it, but there aren't so many of them. And so that's the beauty of having the camera with the AI to be able to identify that at an early stage. And so that means that actually, you know, if you start mobilising the bin lorries, let's say, with the cameras, suddenly you're getting this update every two weeks.

[00:12:02] Suddenly you're being able to identify issues as they're beginning to form, as opposed to once they've formed. And then by intervening, you're able to do that at a much more cost effective way. So suddenly the use of your budget, as opposed to being reactive and responding to something that's created, is now being used in a proactive way to prevent the issue becoming major and costly before it can, basically. One of the things I love about what you're doing here is we hear a lot around autonomous vehicles from companies like Waymo,

[00:12:31] but far less about whether the roads themselves are ready for them. So what does a machine need from road marking signs and infrastructure that a human driver might not simply compensate for? Where are today's biggest blind spots from what you've learned here? Oh, interesting, because the ambition of a lot of the autonomous vehicle creators is not to rely on the infrastructure,

[00:12:55] is to create systems that are as robust as possible to situations when the infrastructure is not as it should be or as expected, much like us as human drivers. However, we do see situations where sometimes things don't quite match. One particular case is in the road markings. Many of the roads, the latest technologies, even on the road already, like ADAS technologies,

[00:13:22] where you're having lane keeping assist systems, leverage a camera looking at the white lines on the road to determine if the vehicle is in the lane or not and then correct accordingly. If those lane markings are not maintained and kept up to code, those algorithms can degrade, let's say, the performance of them, and that can lead to problems. We see through some of the work we do, particularly in the Netherlands with the Dutch road authority,

[00:13:51] that's a particular focus for them to say, how can we leverage the data from systems like universes, but also the cars themselves to say, when have these road markings degraded sufficiently? The performance lane keeping assist systems has degraded and therefore we need to do something about that. So I can see that it's not so much about what the vehicles need, but it's more about creating the virtuous circle where the roads are created for the cars,

[00:14:19] but then the cars are back for the maintenance of the roads as well, and all together create a safer environment for us all to operate in. Your agentic AI might not be secure even with real-time data and proper guardrails, but Denodo makes sure your business has every avenue covered. By placing all your data platforms under one AI data layer,

[00:14:44] your business can reach semantic consistency safely and securely. So get your agents on the same page by visiting denodo.com, and you can learn more about how to start trusting your agents to make business decisions. But now, back to today's guest. And I've been fortunate to have a ride in a Waymo in California, those long winding roads in the US, something we have much less of on European roads.

[00:15:13] And Waymo and Wave represent quite different philosophies around autonomous vehicles. So from your background in autonomous navigation, what can we learn from those approaches? And do you think one is better suited to the, let's say, messy reality of European roads? It's hard to say, I must say. You are right, they have adopted a slightly different philosophy. I mean, certainly Waymo has been at it for much longer,

[00:15:38] and have got that approach from where they've decided to have a much more diverse range of sensors. So the LIDARs plus the cameras plus the radars. They do a lot of work to pre-map the environment and then maintain that map. So when the car operates, it's operating with reference to a map that was created before. Wave and indeed others like Uber, they've adopted a slightly different approach where they said it's just too expensive to create that map in the first place.

[00:16:05] Even if we could create it, we couldn't maintain it. The world changes too fast for us to maintain it in a cost-effective way. So we need to create technology that can adapt to the world, that doesn't need a map, that can just, like you and I as humans, operate in a world where we see a situation and using the innate abilities that we have and the experience that we've gained over the years to operate in an effective way.

[00:16:32] And that's the approach Wave has gone for and is now making, it seems, sort of a lot of success and beginning to deploy in London. I think the jury's out as to which one is going to prevail, which one is going to be more successful. I think Waymo has managed to do that. I think they're the only company who have managed to drive sufficient miles where the statistical significance of their safety results is now for all to see.

[00:16:58] They've clearly created a system that is safer than us as human drivers. And that's to their credit. And they've proven that now. Whether they can sustain that in a cost-effective way and keep the business without the might of Alphabet behind them remains to be seen. But it's an impressive result so far. And I think Wave is trying to emulate that. And it feels like an exciting time for you guys as well.

[00:17:26] Before you join me today, I was doing a little research and reading that Pirelli recently invested in universes and you're bringing together tyres, vehicle sensing and AI-powered infrastructure intelligence. So on that side of things, what becomes possible when information from the vehicle, whether it be the tyre and the road environment, all starts working together rather than in fragmented existing separate systems out there? Yeah, it's a super exciting time.

[00:17:53] And we joined forces with Pirelli actually quite 18 months or so ago. We began discussing some collaboration, but then a more deeper collaboration, let's say, came to fruition earlier this year where they secured a 30% investment in universes. And so they really believe in the journey we're on. I think the philosophy they've adopted is that the tyre applied to the automotive industry, of course, is their core business.

[00:18:18] They believe with the latest connected tyre technology, what they call a cyber tyre, that actually the data that it can be collected from a tyre with sensors embedded in the tyre structure itself enables new insights, partly for the car itself, but also for the infrastructure on which the car is driving. So their approach is to take the tyre, work with automotive, but also new business like the road environment. Universes has the same approach, but we do it with a camera.

[00:18:47] We work with the automotive world, but we also work with infrastructure. And the philosophy is actually camera plus tyre is better than just tyre or just camera. And so in the end, what you have is in the automotive world, a tyre where the contact patch is very well understood, which provides the grip. You have a camera looking forward, which can inform the car what's coming down the road, so they can anticipate as opposed to just sensing and respond.

[00:19:12] And on the flip side, you're also able to detect much more accurately things like surface degradation of the road. So not only spot the pothole, we're talking about beginning to form, but also feel the road in a sense through the tyre that gives a much deeper level of insight. And so what you see is these different sensing modalities coming together. One is more haptic, feeling the road. One is more remote, looking at the road ahead.

[00:19:41] And the different time domain, the different sensing domain gives a much richer picture. I think then all processed with advanced algorithms enables new levels of safety, new levels of performance, and yields a lot of benefit for the customers we're talking to. And there's also a fascinating, much bigger picture here, bigger idea, where millions of vehicles in the future that are already travelling on our roads every day

[00:20:07] will effectively pass infrastructure problems that governments need to know about. So if we look beyond potholes here, how far do you think we could take this idea of turning existing vehicles into distributed intelligence networks for cities? And what kind of safeguards around privacy and public trust need to accompany that? I suspect it's a question you get a lot. That's probably, first of all, people get excited. The next question is around privacy and public trust, right? Yeah, it's critical, of course.

[00:20:36] But on the plus side, I think when you talk to these road authorities, they have these vast networks of roads that they're responsible for. Getting information about those roads is difficult, expensive, and time-consuming. And actually now we're beginning to see these vehicles operating on their network that are equipped with all sorts of advanced sensing and processing. Those vehicles are processing that data to create these so-called digital twins, digital representations of the world,

[00:21:04] so that they know how to behave in that world and can behave autonomously. And our premise is actually that information is incredibly valuable if you can extract it from the car for the road authority to understand what's the current state of their road network. And so we're starting to do that now already with some of the OEMs that we're working with, particularly in the Netherlands but in other territories too, focused on particular asset classes,

[00:21:32] so surface condition, traffic signs, streetlights, all with a view to helping those road authorities understand an inventory, a basic inventory of what assets are out on the road network already, what health, what state all of that inventory is in, and then what actions they should be taking to improve and upgrade and maintain that inventory. And so this is a step change in insight that these authorities will now have, and a step change in efficiency,

[00:21:59] a step change in budget usage, if you like, or cost reduction. And we anticipate also a great update in SEKI too. So that's the positive, that's the benefit, that's what you really get out of this. On the sort of cautionary side, all of this data being collected by these vehicles, it contains potentially anyway a lot of personal information. These vehicles are taking pictures of other cars, the license plates. They might even have pictures of people in those cars or people on the side of the road.

[00:22:29] So it's something we take very seriously here at Universe is it's very important for us that actually when we collect data from a camera, we are not looking to use that as an image to collect personal data. It's simply the infrastructure. Any personal data we collect is an unfortunate byproduct that we treat very, very carefully. We have algorithms to detect faces, detect license plates with an explicit purpose to blur them out so they are removed. We delete the original image,

[00:22:55] and it's only the image that's been cleansed, if you like, from the personal data that we then retain and process for our customers. Well, as an eternal optimist, I'm hoping that a UK transport secretary or somebody listening or government officials will hear these stories of like the Swedish council you mentioned there reducing potholes from 3,000 to 900 in just six months. And listening to those stories

[00:23:23] of how councils are seeing big results in both time and money, maybe they're inspired to find out more about universes and the work that you're doing. Where can people listening find out more information? So the best way in is through our website. It's universes.com. Hopefully we can put a post maybe on the podcast. And that's a great way to find out about what we're doing, how we're leveraging the data from cameras to help that. In the UK already,

[00:23:51] we've worked with a few local authorities, but primarily with national highways, a big focus has been the streetlights. Uh, on the, what they call the strategic route network. Um, it's about 7,000 miles of M road, uh, motorways and A roads, uh, helping them to, as I talked about getting inventory of the streetlights and ultimately reduce their electricity bill for powering those streetlights, which measures in tens of millions of British pounds per year. And we're able to reduce that substantially. So it was a great result to help them.

[00:24:21] Well, it really is an inspiring story from your personal founder story to today, how the universe's technology works, the real world impact that you're seeing, the kind of issues that these cameras are picking out. So it feels like a really exciting future ahead for you guys. So I will include links to everything. If you've got any videos or anything, you'd like me to embed to the blog posts that will accompany this episode, I will add that too, but more than anything, thank you for shining a light on this. Really appreciate your time. No, really. Thanks for having me. It was a great pleasure to be here, Neil.

[00:24:51] And absolutely. I've I've got more than a lot of material to share whenever you, you'll be inundated now, I'm afraid, but, but yeah, got plenty of stuff. I think Jonathan's story today shows what happens when cities stop treating road data as a five-year snapshot and start using it as a regular operating signal. Think about it, a waste collection truck making its normal rounds every day and every week of the year

[00:25:18] that can also identify damage on the roads early, give road teams a priority list of repairs automatically and potentially prevent a small crack from becoming a massive expensive pothole. And that seems to be a far better use of an existing vehicle than simply giving it another dashboard. Nobody wants that. And I think the wider lesson here reaches far beyond roads. Data creates value when it arrives frequently,

[00:25:46] connects to a clear decision and respects people captured along the way. So I'll include the links to everything we talked about today. Love to hear from you. And if your city, if your council could make use of this technology, and if you're going to check it out, as always, techtalksnetwork.com, let me know. Could the vehicles already moving through your city become its most useful maintenance network? Please check them out. Let me know your thoughts.

[00:26:15] If we can rid the roads of potholes by simply adding cameras to council vehicles, that would be a huge step forward. But that is it for today. So thank you for listening, as always. Bye for now.