3119: Open Source Innovation: The ProxySQL Story

[00:00:04] In this ever-evolving world of database management, ensuring efficiency, scalability and reliability

[00:00:11] is no easy feat. Probably something that we're all guilty of taking for granted, especially when we

[00:00:18] look under the hood at the increasing complexities of modern infrastructures. Well today we're going

[00:00:25] to explore how frustration with traditional tools led to the creation of a game-changing open source

[00:00:31] solution. So today I want to invite you to join me in uncovering the story behind innovative approaches

[00:00:38] to database management and how open source principles are maybe transforming the field and dare to look into

[00:00:46] the future and see what it holds for high performance and highly available systems. But to do that I need

[00:00:53] to introduce you to today's guest. So enough scene setting from me, let's get today's guest onto the

[00:00:59] podcast now. Can you tell everyone listening a little about who you are, what you do?

[00:01:03] Hi, I'm Jesmar. Jesmar Canao. I'm the CEO of ProxySQL. In the company I'm wearing several hats right now,

[00:01:12] from sales to customers relationship, to actually taking care of the team, but also dealing with

[00:01:19] blockers and still also pretty busy with hands-on with several customers, sometimes even giving

[00:01:26] database administration words. I was born in Malta. I grew up in Sicily, hence my accent. I don't know if

[00:01:32] you can hear it. I'm in IT for more than 20 years. I was born as a system administrator,

[00:01:39] but later on I was introduced by my brother into MySQL. So I focused and just took the road of MySQL

[00:01:48] administration only. I worked several years around as MySQL DBA for gaming companies to also

[00:01:57] companies in e-commerce, till when I joined René in this adventure with ProxySQL. From there is history.

[00:02:07] I've got to ask, I mean you said you've spent a lot of time in IT, database administration,

[00:02:12] and I know there's a passionate community and also business leaders will take this subject so

[00:02:17] seriously. So I'd love to hear a bit more about the story behind ProxySQL. What were the key

[00:02:22] challenges or pain points for DBAs that maybe you experienced that helped lead to that creation?

[00:02:29] There's got to be a story there somewhere. Yeah, that is a fun fact. ProxySQL was born

[00:02:35] as the result of the frustration of a DBA. René started working on it as a side project. Actually,

[00:02:44] initially he didn't want neither to create a new product. He was facing some issue with loads,

[00:02:50] load balancing and whatnot. And at the time there was a particular software which now is also still

[00:02:58] owned by Oracle, let's say, which was MySQL proxy. But even at that time it was badly maintained and

[00:03:06] René was like, okay, let me see if I can do something with it. If I can push some code,

[00:03:12] if I can work on it and make it better. It was a can of worms. So he said, okay, better to just start

[00:03:19] a new software from scratch. And that was it. He basically started working on ProxySQL. Initially,

[00:03:27] as I was saying, as a side project, slowly, slowly, he just into it, all the pain points that he

[00:03:33] has lived as a DBA. So from query routing to load balancing, but also, I don't know, query writing,

[00:03:43] query firewalling, also read-dry split between the source writer and the replica,

[00:03:48] and making sure that if a failover happened because MHA or orchestrator triggered,

[00:03:55] traffic keep flowing without having the need of the DBA or the DevOps team to redeploy code

[00:04:01] or do something else. So yeah, that was the pain point that we tried to fix, basically.

[00:04:08] And I love that, Larry. It's such a great story because you could have easily just sat there,

[00:04:12] spent the rest of your days complaining, becoming bitter and bitter day by day. But you actually

[00:04:17] decided to try and solve this problem and empower database teams, database teams like the ones that

[00:04:23] you were working in there. So just to introduce everyone to ProxySQL, can you tell me a little bit

[00:04:28] more about how it empowers database teams to ultimately manage MySQL infrastructure more

[00:04:34] efficiently, whether it's in cloud or on-premise? For anyone listening hearing about you for the first

[00:04:40] time, could you expand on that? Absolutely. ProxySQL is a very lightweight piece of software

[00:04:45] that sits in between the application and the database of the client, the customer, the company that wants to

[00:04:53] use it. The company takes care about the installation and the configuration of it. So we have no,

[00:05:01] like it's not a third-party software or whatever else. You just take it, download it, install it. It's

[00:05:07] fully open source. So you can do wherever you want and you can install it wherever you want. So on-prem,

[00:05:13] on the cloud, on a VM, on the same machine where the application is running or on the database,

[00:05:21] literally you can install it everywhere. As long as there is Linux, you can run it. Even inside

[00:05:26] Kubernetes, we have several of our customers that use it inside Kubernetes, for example.

[00:05:33] So the point is that the database teams or whoever takes care of the database, the way how they are

[00:05:43] empowered, they literally love it. Because consider that before ProxySQL was out, whatever the application

[00:05:50] was sending, the database administrator couldn't do anything because there was no way to block the

[00:05:58] traffic. There is no way to rewrite the query. There is no way to just say, okay, this query,

[00:06:05] send it somewhere else. There was no actual way to do that. And so I come from a place where

[00:06:15] the software developer will always tell you, oh, is the database fault? And that is the frustration

[00:06:23] that actually the DBA has all the time because the DBA look at the DBA, look at the stats monitoring and

[00:06:29] says, oh no, it's not the database fault. It's what you are sending from the application.

[00:06:33] And most of the time, the application owner says, oh, it's what the ORM is sending me. Oh,

[00:06:40] okay. So yeah, we are in a kind of loop. ProxySQL instead, being something in the middle,

[00:06:50] can actually empower the DBA that just monitor the traffic, see the query, says, okay, this one,

[00:06:56] no, no, let's send it to the replica, for example. This one is bad as it is written right now.

[00:07:02] It's bad. Do you really need, for example, this order by? No. So why is there? Let's remove it.

[00:07:08] So this type of things are the way how the database administrator is empowered. At the same time,

[00:07:16] you give the possibility to actually shard with minimum knowledge being given from the application

[00:07:24] point of view. That means that you can distribute load across multiple clusters or even just distribute

[00:07:32] the load across multiple replica nodes instead of just hitting the writer node all the time.

[00:07:38] It's funny, I was reminiscing there listening to you talk. I've been in those IT time kind of

[00:07:43] situations. It's like the Spider-Man meme, everyone pointing at each other. It's the network guy.

[00:07:48] The network guy loses it. It's not the network, it's the application. It's not my application says

[00:07:52] that the application guy, it's the database and everyone's pointing at each other.

[00:07:57] Exactly. And basically you have now ProxySQL that can actually pinpoint and say, okay, you think

[00:08:02] is that? Let me try to fix in this way. Oh, you see? Fixed. But also remember, even this type of

[00:08:09] fix of queries, we have had several customers in which we were just fixing the query immediately

[00:08:16] instead of having the developer to have to search in the code where it's being generated and fix it

[00:08:22] there. So you can remove that performance degradation in the matter of one minute instead of half an hour

[00:08:32] of development looking for that query. What also stood out to me about you guys is a time where so

[00:08:38] many companies are moving towards a more closed source solution. What was it that made you choose

[00:08:44] to make ProxySQL open source and how does that shape your business model? Absolutely. That's

[00:08:51] actually a very great question. The point is that we are very passionate about open source. ProxySQL

[00:08:57] has to, had to be an open source product simply because first of all, MySQL is an open source product.

[00:09:06] We believe in that philosophy. René comes out from MySQL AB. He used to be inside MySQL,

[00:09:12] so has ingrained into him that model of open sourceness. And at the same time, we want that people can

[00:09:20] install it in any environment without having to consider how much, how big is their environment or if I can

[00:09:28] pay for that licenses or whatever. You should be able to use it everywhere. And also being completely

[00:09:36] transparent in terms of being open source, you can go and look at the code. It kind of gives trust

[00:09:45] to the final users, which again can be from the small customer to actually very large, even financial

[00:09:53] companies that for them anything is a problem. Like even before we were talking, for example, about a call

[00:10:02] to an external rep or whatever else can be seen in such a bad way from financial companies.

[00:10:10] Yeah.

[00:10:11] So we believe that code should be completely transparent. People can just rely on the code itself

[00:10:19] and that's it. That's when basically we change the business model on how we work. We do not work

[00:10:27] on licensing, we work on support. So we give to our customers support, training, consulting on the product itself.

[00:10:38] And then because we are DBAs from ages, we also offer consulting on MySQL database in general as well.

[00:10:47] And in this way, basically, you have companies that really just trust the project and they are happy to

[00:10:56] pay this type of solution simply because one, it guarantees that the project will keep on going.

[00:11:02] Two, for them is that I would say, I want to say kind of insurance policy because if they have any issue,

[00:11:10] they have a door to which come and knock and ask for help or new features because we have also customers

[00:11:18] and companies coming to us asking for new features to be added into proxy SQL because they want to have,

[00:11:25] I don't know, new visibility of particular stats or because like I can mention, for example, DigitalOcean just not long ago.

[00:11:35] We then work together to add the proxy protocol into proxy SQL. So yes, that's also something that we offer and people trust us at that point.

[00:11:47] So incredibly cool. And we're recording this at the IT press tour in Malta and talking about open source, you had quite a few questions today from people questioning the brave, bold and trusting move of going open source.

[00:12:02] So what would you say are some of the biggest challenges you faced as an open source company and how have you overcome them?

[00:12:08] So being an open source company comes absolutely with a lot of opportunities. Imagine people that contribute to the project and so on and so forth,

[00:12:20] but obviously there are a lot of challenges coming from it. So one, for example, of the biggest challenges that we faced was to actually have the open source community

[00:12:33] that might have demands for building a sustainable product and a sustainable business. So balance between the two.

[00:12:41] These challenges obviously are on the funding of the development itself and hence the business model that we were discussing before.

[00:12:52] But at the same time, when a product is open source, a lot of companies, they just say,

[00:12:59] it's a tool for free, it's a no, how to say, it's not a big deal. It's something that doesn't bring a lot of on the plate.

[00:13:09] The reality is that the proxy sequel brings a lot on the plate and being self bootstrapped, self-funded, always for us is important to have enough liquidity to being able to support the full team.

[00:13:24] So that's actually a huge challenge for us.

[00:13:28] And that's the reason why the growth in the company has always been steadily and slow.

[00:13:34] But at the same time, because we want that the product to remain open source, what we are trying also to work on is in building a possible community and the possible foundation behind the proxy sequel,

[00:13:49] with also clients staring a little bit on what new features might be wanted to be delivered in the proxy sequel itself.

[00:14:03] The other problem is competing with other projects that instead are a mix of open source, closed source.

[00:14:12] It's pretty hard because obviously if you are closed source, you have also investors behind you.

[00:14:19] It becomes hard to compete with them in terms of spending power also for marketing or for publicity or to be outside there.

[00:14:28] Then the other problem that we also faced is to build a community around it.

[00:14:35] My SQL community is a very vibrant community.

[00:14:39] And when proxy sequel came out, it was immediately embraced because it finally solved an issue that everyone was failing facing.

[00:14:51] But obviously, again, being small, that means that we have to push out a lot of documentation, push out a lot of other papers and have the community trying to participate into the discussion.

[00:15:06] That is another issue that we faced as an open source.

[00:15:10] And you mentioned competing there.

[00:15:13] And I'm speaking to more and more startups that have got distributed teams because they can get the best talent from anywhere.

[00:15:20] And obviously, you said at the beginning of our conversation, you're from Sicily, you're working and living in Malta right now, and your team operates remotely.

[00:15:27] So how has that approach benefited the development and support of proxy sequel?

[00:15:33] And what challenges have you had to navigate with fully remote, fully distributed teams?

[00:15:37] Well, let's start with the benefits.

[00:15:42] The nice thing of having a team that is fully remote basically allows you to access to a larger pool of talent.

[00:15:52] If you are just based in Malta or you are just based in the UK or whatever else, you can rely just on talent that you find or that you can convince to move to these countries.

[00:16:04] When you are instead fully remote from day one, that is your philosophy.

[00:16:09] Then you can access a pool from everywhere.

[00:16:13] And that's a reality.

[00:16:14] We have people working with us that are from Malta, some that are from Hungary.

[00:16:20] We have in Brazil.

[00:16:21] We have in Greece.

[00:16:23] We have Spain.

[00:16:25] We have people scattered around.

[00:16:27] Also India, Pakistan.

[00:16:29] We have people scattered around a little bit everywhere.

[00:16:31] That helps not only because they actually cover better time zone for issues with customers and whatever else,

[00:16:41] but also there are different cultures, different mentality, and that helps in providing a better solution and a better software sometime.

[00:16:51] Because there are different approach on how things can be done.

[00:16:55] Obviously, apart from benefits, we have to discuss about difficulties.

[00:17:00] Because you can imagine when you have a full remote company, you lose a little bit of that touch of the random conversation at the office or whatever else.

[00:17:11] To avoid to have this type of distance, what we try to do is to have always meetings once a week in which we discuss a little bit of the progress of everything.

[00:17:25] We have retrospective.

[00:17:26] We are one-to-one.

[00:17:27] We have also casual conversation.

[00:17:29] Just something happened.

[00:17:30] Okay, let's jump on a call.

[00:17:32] Let's discuss.

[00:17:32] We believe a lot in the personal touch.

[00:17:35] We do not have employees working for us.

[00:17:39] We have people working together with us.

[00:17:41] It's different.

[00:17:42] Then, obviously, what we try to do is to at least once a year to have, like, we try to fly everyone in.

[00:17:50] We take advantage of the FOSDEM.

[00:17:53] I don't know if you are familiar with it.

[00:17:56] FOSDEM is the largest open source conference in Europe.

[00:18:00] It's done in Brussels.

[00:18:01] They usually have a MySQL dev room before there is also the MySQL days sponsored by Oracle.

[00:18:07] So we try to fly everyone in during that time.

[00:18:11] Then chocolate and beer are a good glue in a human relationship.

[00:18:15] Of course, we're recording this in December.

[00:18:17] We're now weeks away from 2025.

[00:18:20] So many big changes, lots of hype around AI, etc.

[00:18:23] But from your perspective, where is the database world heading?

[00:18:27] How is that evolving and are there any trends that you see shaping the future of database management?

[00:18:33] You must be seeing a lot at the moment.

[00:18:34] What we are seeing lately in the last few years is that there are several companies that are kind of moving towards the database as a service approach.

[00:18:47] So they are kind of removing the need to deal with the database really and truly.

[00:18:56] We have several open projects, open source projects in the database world that are kind of changing their license.

[00:19:05] Or that they are releasing features just to their enterprise version or on their DBS service solution.

[00:19:17] And things will trickle to the open source eventually.

[00:19:22] Or we have other open source projects that because they are fully open source, there is no one really behind taking decision.

[00:19:31] There is a large committee.

[00:19:33] No one takes decision.

[00:19:34] And the features in the open source takes long.

[00:19:39] And you have a lot of closed source based on that open source companies that flourish all around to just sell you the new features.

[00:19:47] Okay, you want that feature?

[00:19:48] I can sell it to you.

[00:19:49] But you are then stuck with me.

[00:19:51] You have to pay my license and whatever.

[00:19:52] So we have this migration of open source project.

[00:19:56] There is a little bit of tumultuous right now going on.

[00:19:59] And even, for example, sometimes also for simple HA solutions that become kind of proper, like they are closed source, the solution for HA.

[00:20:09] It's something that send me kind of nuts sometimes.

[00:20:13] And this, what happens is that drives away the figure of the DBA from companies.

[00:20:22] I've seen a lot of companies that if you deal with them, there is no DBA in-house.

[00:20:29] Really?

[00:20:30] There is an infra team.

[00:20:32] There is a DevOps team.

[00:20:34] But the DBA, it's slowly, slowly being pushed out from companies.

[00:20:41] What happened is that then these DBAs are moving and working for companies that offer DBA as a service then.

[00:20:48] And it's a bit of a weird word right now.

[00:20:52] Yeah.

[00:20:52] But at the same time, I say it's good for consultants.

[00:20:56] Yes.

[00:20:57] Wow.

[00:20:58] I think we're going to have to get you back on next year for a dedicated episode just on DBA as a service.

[00:21:03] But looking forward to next year and everything we've talked about, how is ProxySQL?

[00:21:08] How are you evolving to stay ahead of the needs of modern database infrastructures, especially with the increasing demand for high performance and availability?

[00:21:17] Hold on, what's next for you guys?

[00:21:19] So actually having the figure of DBA that starts to disappear, it's actually good for us, kind of.

[00:21:28] Yeah.

[00:21:29] Because what is happening is that ProxySQL, already out of the box, have the possibility of connection pooling, multiplexing,

[00:21:35] that solve immediately some pain points that companies might face.

[00:21:39] And because ProxySQL is very lightweight, it's very easy to configure, at least to make it run immediately, can be done from anyone.

[00:21:48] You don't need an actual DBA to do that.

[00:21:51] Yeah.

[00:21:52] And it's already a seller point for companies to be engaged in the use of ProxySQL.

[00:21:59] But obviously, then they dig in, they like the product, they want it to be doing way more.

[00:22:06] So at that point, they come to us to ask for consulting or support in which they can ask for any question.

[00:22:13] You know, that's good for us.

[00:22:16] That's why I'm seeing ProxySQL still having a bright future ahead of it.

[00:22:22] And the other point is that ProxySQL being implemented, make sure that the environment reach very, very high availability.

[00:22:34] Like we are talking about even 99.9999%, you know, that zero, zero that people love.

[00:22:42] Actually, you can reach it because it's true that you are not going to have incident every day,

[00:22:47] but you want that that incident doesn't last half an hour, doesn't last.

[00:22:53] Like someone has to wake up, open the laptop, connect to the VPN, check what happened and then act accordingly.

[00:23:00] ProxySQL can react without having anyone there.

[00:23:05] So a possible downtime that in a normal structured environment can take 15 minutes can be resolved instead in 30 seconds if things are configured properly.

[00:23:17] So then the other thing that actually we see for ProxySQL is the fact that we are finally digging deeper and deeper

[00:23:28] into the Postgres protocol.

[00:23:31] So we were a bit afraid, how to say sometime, to start dealing with it because it was a completely new thing for us.

[00:23:41] We are MySQL DBAs and we know MySQL inside out, but Postgres has a protocol on its own.

[00:23:49] And also the internals of Postgres are different.

[00:23:52] But we were receiving a lot of requests from the Postgres community, like to have a software that like, can you do a proxy SQL?

[00:24:02] What you do for MySQL, do that for us, for Postgres.

[00:24:07] So slowly, slowly, we kind of onboarded this idea.

[00:24:11] And around one year and a half ago, we finally say, OK, let's try.

[00:24:17] Around one year and a half ago, we had a small prototype.

[00:24:21] We never released it, neither in alpha.

[00:24:23] We never went out.

[00:24:25] It was just point selects, no transaction, nothing.

[00:24:28] Just let's see if it works.

[00:24:29] It worked.

[00:24:31] And on that, then we said, OK, let's actually build something better.

[00:24:36] In October, so not last month, but two months ago, we finally released a version proxy SQL 3.0, which is released in alpha mode, which actually support Postgres protocol.

[00:24:52] So support at the same time MySQL protocol and Postgres protocol.

[00:24:56] This month, in December, we are going to release a beta version.

[00:25:01] Is it going to be production ready?

[00:25:04] Maybe not.

[00:25:04] Yeah.

[00:25:05] But already for non-critical environment, not critical application can start to be used.

[00:25:12] The reason why we released the alpha and the reason why we are going to also release the beta is because we need the community to tell us if it's OK or not, what they want.

[00:25:23] Because for us, it's very important, the feedback from the community.

[00:25:27] So far, we received a few feedbacks.

[00:25:29] We would like everyone out there to play with it and tell us more.

[00:25:34] But yes, that's it.

[00:25:37] This is our future, I would say.

[00:25:41] Well, you'd like everybody to give it a try, play with it and give you some feedback.

[00:25:45] So for anyone listening, maybe they want to take you up on that challenge.

[00:25:48] Where would you like to point everyone listening to not only give it a try, but also contact you or your team?

[00:25:53] Where would you like to point everyone listening?

[00:25:55] So they can go on our website, which is proxysequel.com.

[00:25:59] Or they can come on, they can look for us on LinkedIn.

[00:26:03] Like they can look for a proxy sequel on LinkedIn.

[00:26:06] And absolutely, they are going to find me, Jesmar, or even René.

[00:26:11] Jesmar can know or René can know.

[00:26:13] And we are always pretty active even over there.

[00:26:16] So if you need anything, just drop a line.

[00:26:19] Even on our website, there are several forms where you can drop a request and you can tell us more.

[00:26:25] So we have also, you can look for the project and you can find their issues if you think that there are certain things missing.

[00:26:36] And we can look into it.

[00:26:39] Awesome.

[00:26:39] I'll add links to everything to make it nice and easy for people to find you.

[00:26:43] We covered a lot today.

[00:26:45] Talk about your origin story at proxy and sequel.

[00:26:47] This solution that you created to empower the team taking care of databases around the world.

[00:26:54] And also the challenges and benefits of managing a remote team, where the database world is moving, and the future of proxy sequel.

[00:27:01] I'd love to stay in touch with you to find out more about how next year goes and how that evolves.

[00:27:05] But thank you for joining me today.

[00:27:07] Thank you.

[00:27:08] Thanks a lot.

[00:27:09] So as we close today's discussion, I think it's clear that rethinking database management can unlock immense opportunities for organizations that are navigating hybrid and cloud environments.

[00:27:22] And open source solutions not only empower teams, but also foster innovation and adaptability in this rapidly changing tech landscape.

[00:27:32] But what are your thoughts on some of the challenges and opportunities in modern database management that you're seeing right now?

[00:27:39] You've heard from me.

[00:27:40] You've heard from today's guest.

[00:27:41] We've had a conversation about some of those challenges, but I'm sure you've got many more too.

[00:27:46] So please email me now, techblogwriteroutlook.com, LinkedIn, Instagram, X, just at me or see Hughes.

[00:27:54] But as always, thank you for taking the time to tune in to Tech Talks Daily.

[00:27:58] And let's stay connected.

[00:28:00] And we'll catch up again in tomorrow's episode.

[00:28:03] Hopefully I'll speak with you all then.

[00:28:06] Bye for now.