Cognichip CEO Explains the New AI Race Happening Inside Semiconductor Design

[00:00:00] A quick thank you to NordLayer for supporting the podcast and helping me make these daily conversations possible. And if you are listening and you're responsible for security or IT, you will know the reality. The reality that most of your risk now sits inside SaaS apps and browser activity. That gap is exactly what NordLayer is addressing with its new business browser.

[00:00:24] So instead of bolting security on from the outside, it builds it directly into the browser itself. This means you can control access, monitor activity, enforce policies and reduce shadow IT all from one single place. And most importantly, it does it without adding deployment headaches or complex onboarding.

[00:00:48] You get things like browser-based data loss prevention, SaaS access control and zero trust browsing, but delivered in a way that your team can actually use. So if you've been trying to simplify your stack while improving visibility, please check it out at nordlayer.com slash browser. But now it's time for me to introduce you to today's guest.

[00:01:15] What if the real bottleneck holding back AI innovation is not the software that everyone's talking about, but the hardware quietly struggling to keep up behind the scenes? Well, today's conversation takes us right into the heart of that challenge because I'm joined by the CEO of Cognichip. And let me tell you, he is quite a special person.

[00:01:38] He's a two-time semiconductor IPO founder with more than two decades of experience building and scaling companies in one of the most complex industries on the planet. From helping bring DSL broadband to the world to accelerating data center connectivity, my guest has been a part of the infrastructure that big fan over here. Huge inspiration to me. What a story this is.

[00:02:04] But apart from that, one of the things that stood out to me too in our conversation is not just his track record, but it's the decision to step back into the arena and build again. Almost reminds me of that scene in Godfather 2. Just when I think I'm out, they pull me back in. But seriously though, after time spent investing and watching the rise of AI from a different vantage point, he saw a deeper issue that many outside the semiconductor world would rarely consider.

[00:02:34] And that is software is now evolving at extraordinary speed. But the chips powering the software, well they're often designed years in advance. And that is creating a growing disconnect between what AI demands and what hardware can actually deliver. And it is this realization that led to the creation of something entirely new.

[00:02:57] So CogniChip is developing artificial chip intelligence, which is a physics informed AI model designed specifically for semiconductor design. And it is this ambition that is not only bold, but it will reduce the time it takes to build a chip from years to something far faster. But it also cuts costs dramatically, opening the door for a wider range of innovators to participate in helping shape the future of hardware.

[00:03:27] So I want to try and get a greater understanding of this and explore what is fundamentally broken in chip design today. Why the next phase of AI depends on rethinking how chips are created. And how the global supply chain shifts like Pax Silica could actually reshape the economics and pace of innovation. And there's also an incredibly deeply human layer to this story.

[00:03:53] From talent shortages to the change in nature of engineering itself and that inspiring career that he's enjoyed. So how do you rethink the industry that has been evolving for decades? And what happens when AI starts designing the very chips that power it? Well, enough for me. Let me introduce you to him right now. So a massive warm welcome to the show. Can you tell everyone listening a little about who you are and what you do? First of all, my name is Faraj Al-Ai.

[00:04:22] And I am a bioeducation and electrical engineer that went to business school and got the bug of business and started building businesses. I was fortunate enough to build two semiconductor companies and take them public over the last, you know, 25, 26 years.

[00:04:47] One was Centillium Communications, which brought to the world the DSL technology for broadband access. And the other one was Aquantia, which I took public on New York Stock Exchange, was actually speeding up data center connectivity. After the second one I took public, I decided to go into the investment business.

[00:05:11] So I started the firm called Can Do Ventures with one of my co-founders from my company building experience. And we started to invest in software companies and a lot of AI. This was right around the time when AI was starting to get hot. And so that was quite, quite interesting. But, you know, once you have this entrepreneurial bug, you've got to get back in.

[00:05:36] And playing from the sideline is not quite as satisfactory as it is being in it. And so I jumped in and started this company called CogniChip, which, you know, the pain point of which were ones that were all too familiar to me building these two semiconductor companies and taking them public.

[00:05:57] And so I focused on building what we call artificial chip intelligence, which is really a physics-informed foundation model that is purpose-built for semiconductor designs. Right. And so if you're in this semiconductor business long enough, you realize that it is quite complicated to make these chips. It takes a very long time.

[00:06:24] And if you want to build intelligence that does the design for you, you need to actually train them in this trade, so to speak. It's not like poetry and it's not like writing a memo. It's actually formulating billions, tens of billions, 100 billion transistors, every one of which has to do precisely what it's supposed to do.

[00:06:47] So it's a very difficult task and it requires really the hard work of building models that can put that together. Wow. It's an absolutely incredible journey that you've been on. Hugely successful career there and inspiring as well, taking two companies like that public. And as someone that spent, what, two decades or over two decades building and investing in semiconductor companies, I've got to ask, when you look at chip development today,

[00:07:16] you must have seen so many changes over the years. What feels fundamentally broken and why has it become such a bottleneck for AI progress? Because we see different problems appear throughout each cycle. But what are you seeing this time around? Well, you know, the chip business is what, the 60, 70, it's like six, seven decades old.

[00:07:36] And it's gotten progressively more complicated to build these chips because the chips are expected to do a lot more things. They're a lot more intelligent. And so, you know, it's basically today, where we are today, it takes a few hundred million dollars of investment. And it takes three to five years to design a chip.

[00:08:04] Now, what's broken is in today's world, you have software like ChatGPT that scales from zero to tens of hundred million dollars, a hundred million users in a matter of weeks. And if the underlying hardware that is delivering that ChatGP to every user was designed seven years ago, it couldn't have possibly taken into account the demands of these types of applications.

[00:08:32] So that's an example of showing you how fast software goes from standing still to having millions, hundreds of millions of people on it. And it's operating on old software, old hardware. The reason for that is because over the last few decades, we built this essentially manual and sequential way of developing the chip. Bring people with lots of different expertise, very deep vertical expertise,

[00:09:00] hands off from left to right and left to right and so on and so forth. And it's literally like a waterfall workflow, right? And if something breaks, the later that something breaks or the later a bug is discovered, you have to go all the way back and repeat this process, right? So it takes a long time, costs a lot of money, increasingly a lot of money. And so that's a fundamental problem today in chip design.

[00:09:27] And to top that off, a lot of our young people are choosing in science, instead of getting electrical engineering degrees, they're getting software degrees and computer science degrees. So we actually don't even have enough electrical engineering to keep up with these demands, right? So structurally, we have these really three, what I see, big problems we need to solve. Time it takes to make a chip.

[00:09:55] The cost of making a chip, which is becoming prohibitive for a lot of entrepreneurs, frankly, especially first-time entrepreneurs. And then this, you know, systematic and structural talent shortage, which adds to this compilation of problems. So the world needs something different. We can't do it the same way we've been doing.

[00:10:18] And as you said a few moments ago, CogniChip is introducing this idea of artificial chip intelligence, which sounds incredibly cool. But for people outside of the semiconductor world, I want to leave anybody behind here. What does that actually mean? And how does it change the way chips are designed? Yeah. So artificial chip intelligence, or ACI, is an AI system that actually understands how chips actually work at the physical level.

[00:10:48] Because a lot of the time when you apply AI to hardware, like chip design, or like, you know, when you're building robots, as an example, at the end of the day, you are making a physical specimen of some kind, coordinated in a certain way, right? So these are not generic LLMs that can do this.

[00:11:11] So what you need to do is to train these models at that, in our case, at the transistor level and transistor behavior and device physics, so that it can, as it's reasoning through building up the design, it can consider what is the impact of this on power consumption? What is the impact of this on the size? What is the impact of this on performance of this final product that I'm designing?

[00:11:41] So that's a fundamental performance that you need when you do chip design. This is what ACI delivers. And then, of course, you know, we got to also get rid of this serial format that we've had, this waterfall that I just spoke about, right? And move it from a serial format where people hand off work to each other to a concurrent model within the computer world,

[00:12:07] where lots of tasks are being done at the same time and relate to each other so they can land in the right place, provide a solution that actually works and meets the performance that you want in a very short period of time. So we believe in what we've said from the beginning, and frankly, we are, as we've been developing the last two years, are finding that we can actually deliver even better than this in terms of speed. But we've said from day one, we want to cut the cycle in half,

[00:12:36] and we want to reduce the development costs by at least 75%. And what we're finding as we develop these models is that we can actually shrink that time a lot more than 50%. So it's fascinating when you think about the impact of these systems on the future chip design. It's going to change the world the way we know it in terms of hardware-software interaction.

[00:13:05] Wow, that's incredibly cool, especially when building a chip can take years and cost well over 100 million there just to produce. And I'm curious, how does CogniChip realistically compress that timeline and the cost without compromising performance or reliability? Are there any trade-offs here? Yeah, so if you think about the element of time that we talked about, right, is these design iterations.

[00:13:35] So today in the world, we expect each one of these verticals of expertise human beings designing provide a perfect handoff to the next person who is going to take it the next path. And this goes on, by the way, two to three years to finish that design. So when something breaks, when somebody downstream realizes that it's not actually doing what it was supposed to do, it has a bug, it's an integration site, right?

[00:14:03] So, and this iteration cycle, you know, costs you months in development and millions of dollars. What we do with ACI is we collapse these loops into each other where all of these parameters are being done by the compute clusters in parallel with each other and constantly checking with each other. So think about like putting a Rubik's Cube together, right?

[00:14:30] And you just actually are moving at a lot faster speed and in lots of different dimensions at the same time. That's what it does. And that's how we could collapse that time. And because now engineers don't necessarily have to do these tasks by hand, they can move up a little bit higher in the paradigm and become more like architects, right?

[00:14:51] So now you free up these engineers to actually go do other stuff, do other chips and other products and other functions, right? So it's a combination of these two that you get the collapse of time and the reduction in the cost of doing this. And by delivering this to customers today, even as a young company, we're engaged with over 30 enterprise customers

[00:15:18] that are evaluating these products from one angle or another, applying it to their own field. And by the way, these things are, they're not built for just one vertical narrow field. They can do any design task in chip design. Big thank you to Denodo for supporting the Tech Talks network and making these conversations possible.

[00:15:40] Because when your lake house stores the data, the real challenge is getting that data where it needs to go and faster. And your lake house stores the data, but Denodo helps deliver it faster. So with real-time access, built-in governance and a business-ready data marketplace, Denodo can help your teams unlock insights without costly duplication. And you can learn more by simply visiting denodo.com.

[00:16:11] There's also this growing belief that the future of AI will ultimately depend on AI design chips. So it feels like you're in the right time at the right place. But again, for the business leaders listening, the not-so-techies, why is software innovation now constrained by hardware? And how urgent is this problem that we're talking about? This is absolutely urgent. I mean, you hear every day in media about the mega data centers that we need to build

[00:16:39] and all the energy needs that are there. One of the fundamental reasons, to be honest, is that, again, these software are running on chips that were designed seven, eight years ago. And they're ready for use today. And so that design cycle being so long contributes to high energy costs of running a data center. But essentially, what we're talking about is by getting, you know, in the old time,

[00:17:05] if you go back to prior to degenerative AI and these kinds of models, you know, we've been building chips that have been creating faster computers. The faster computers enable people to write bigger pieces of software, which eventually this cycle led to AI having enough, fast enough computers that you could actually implement AI. Now, if we use AI to build the next ship itself, we're closing this virtual cycle.

[00:17:35] So the feedback loop becomes a lot faster. And the products that you design become more specific to actually the payloads, the AI payloads, a behavior of AI. So when you build those kinds of chips, bring them to market faster, they're lower power and lower cost. You keep that virtual cycle running a lot faster. And we get ahead a lot faster. And I was reading, you've also spoken a lot in the past about the need for a new scaling factor in semiconductors.

[00:18:05] So if Moore's law is no longer the guiding force that it once was that I grew up with there, what replaces it now? Well, Moore's law did a lot of good things for us, right? And it's gotten us to where we are today, you know, doubling the transistor density every couple of years. But that curve has flattened, as you just said, right?

[00:18:25] And we can't really have, you know, push physics, have the physics push your rate of performance, right? So what we need, the new lever, in my opinion, is no longer just packing more transistor. It's actually design intelligence. That is really the new equivalent of Moore's law, right? It's design intelligence. How fast can we design things?

[00:18:52] How can we optimize across a full design space? And how quickly we can iterate on these things when new things come up, right? So that design intelligence really becomes the new operative word from my perspective. And ACI, we've set up ACI to deliver on that design intelligence, right? So that you can have the paradigm shift that the industry needs to get to where it wants to go. And that's what ACI has been to learn.

[00:19:23] Now, there's also a lot of discussion right now around Pax Silica and the reshaping of global supply chains. Massive talking point. But from your perspective, as someone right in the heart of this space, what does this shift get right? And where could it create maybe unintended consequences around everything from innovation to cost? It's been a problem. By the way, in the smaller ways, those of us in the semiconductor industry had built that before.

[00:19:50] Every time you had an earthquake in Taiwan, we would all just scurry around trying to figure out how we could get the chips shipped. But, you know, structurally, I think ACIs like the chip ACI in the U.S., a very fundamental approach and directionally correct is to bring, you know, spread out where we actually create these supply chains.

[00:20:17] But, you know, to say I'm going to go build a fab is easier said than that. It takes years to build these things. It takes tens of billions of dollars. And it's growing, by the way, by the year of how you build a fab. So it's not like you can just pick up a fab and just go build another one tomorrow, right? So some of these long-term structural stuff, I believe, is good, is directionally correct.

[00:20:48] Where we fit in this approach is that we provide geographic flexibility in where the chips are designed. And the way we do it is by not just bringing the intelligence to chip design, but we're actually creating an operating system around which teams, maybe in even different places in the world, but certainly all around the world.

[00:21:17] If they know what they want to build, they now have the intelligence, you know, in the palm of their hand, essentially, through their keyboard into a cloud infrastructure that they can actually design that chip, right? So being able to design chips, being able to move workloads around the world using the artificial chip intelligence system, it gives you another degree of flexibility that you can react to certain kinds of changes in our ecosystem.

[00:21:45] And there are so many challenges in the industry right now with geopolitical tensions rising, projected shortage of engineering talent. How do you see the semiconductor industry adapting and what risk should businesses be relying on AI infrastructure? What should they be preparing for? Again, there's a lot of uncertainty here, but the industry always bounces back and is very innovative. But what do you see happening and how do you see this unfolding?

[00:22:13] Well, I mean, you know, we're talking about really fundamentally dependency, right? We have, you know, geographic volatility that you talked about. We have this talent shortage, which, by the way, the two sides are the same risk. You know, you have countries around the world like Japan who've had significant population decline as an example, right? And how do you react to that, right?

[00:22:38] So these dependency on small pool of experts, specific locations in the world that I could do this or that, and these long design cycles, you know, we need to respond to these. We need to be able to respond to these and respond to them fast, right?

[00:22:58] So, you know, from my perspective, what we're doing at CogniChip is a structural response to the entire food chain of chip building, right?

[00:23:13] Now, what we're doing and what other colleagues may do in other aspects of this chip design, we think are structurally needed to be able to get past these types of problems we've been having with supply chain and dependencies and so on and so forth.

[00:23:33] So in this journey, I think, you know, the most at risk are really going to be design teams or companies that take this lightly and want to stick the way they did things for the last 20, 30 years.

[00:23:49] And I think there needs to be a rethinking in every semiconductor company about not just using AI, but how do you use AI and to what extent do you change your paradigm of running your business and running your innovation engine? So I think those are very, very critical items that a lot of folks in the semiconductor industry are looking at. And like you said, this has been an industry that's very resilient, it's very creative.

[00:24:18] So my hopes are very high that we will make that transition into a better place. And I've got to ask, I mean, what excites you about the future? We've talked a lot around the challenges. I mean, have you even thought about the possibility of taking a third semiconductor public in the future? That would be quite something to achieve legendary status there. But what excites you about the future and everything that you're working towards?

[00:24:42] Look, what brought me back into building another company of my own was the excitement around what this AI brings to the table. You know, every time you build a business, you're really capitalizing on some kind of discontinuity in the marketplace, right? And we're in the tech. So this continuity in tech provides opportunities, provides a way for you to see the world in just a completely different way.

[00:25:13] And I got extremely excited when I started to learn about Gen.AI as I was working in my investment time with these young entrepreneurs. And I started to think, well, wait a minute. All the issues that I have been struggling with the last, you know, two, three decades of building semiconductor companies. Could it be possible that I could do it with AI? I could do it with AI. And that was, that's what brought me back. That's what gets me excited.

[00:25:41] I want to be spending my time impacting the industry that has been so good to me and do it in a way that hopefully, you know, will take another generation or two or three in that path. So it's very exciting for me at the personal level. Absolutely love that. Perfect moment to end on.

[00:26:04] But before I do let you go, for anybody listening wanting to carry on this conversation, connect with you or your team, find out more information about Cognichip, keep up to speed with some of the announcements that will be coming out as well. Where would you like me to point everyone? Please point them to cognichip.ai. That's our website. There's a lot of information there. There's a lot of background information, by the way, for those who want to learn more. And then Cognichip on LinkedIn.

[00:26:32] We are pretty prominent on LinkedIn these days. And so you can actually follow us on LinkedIn as well and follow the news. Wow. Well, we covered so much in a short amount of time today. I, for one, am walking away inspired by your journey, not just in the past and the achievements there, but where you're going, where you're taking this. So I would add links to everything that you've mentioned.

[00:26:57] I'd encourage people listening to go and follow some of those pages and find out more information, maybe carry on that conversation. But more than anything, thank you for joining me today, sharing your story. Really appreciate your time. Thank you, Neil. Thanks for having me. It was a pleasure. After listening to Faraj Al-Ai's journey, it's hard not to come away with a sense that we are standing at a turning point in how technology is built right from the ground up.

[00:27:22] And this idea that the next wave of progress won't come from squeezing more performance out of existing systems, but from rethinking the process entirely really struck a chord with me. Moving away from slow sequential chip design towards something faster, more collaborative and driven by intelligence at the physical level is a massive step forward. And I think it changes the conversation from incremental improvements to something far more ambitious. And that is what excites me.

[00:27:52] And there is also a broader perspective here that goes way beyond semiconductors. Questions around supply chains, talent shortages, geopolitical shifts. These are no longer abstract discussions. They are shaping the cost, availability and direction of the technology that businesses rely on every single day. So whether it is Paxilica or the growing demand for AI infrastructure, the ripple effects will be felt across every industry.

[00:28:20] And then there is the personal element. After decades of building companies and even stepping into the world of investing, my guest chose to return to build again because he saw what was coming. And that decision says a lot about where the real opportunities are emerging and where the biggest problems still need solving.

[00:28:42] So if you want to learn more about what Faraj Al-Ai and his team are building, you can head over to CogniChip's websites or follow the updates on LinkedIn to stay close to the progress that they're making. But for now, I'll leave you with this. If AI is set to shape the industry and the chips behind it determine what is possible, are we paying enough attention to the foundation that everything else is built on? Food for thought.

[00:29:08] So please pop by techtalksnetwork.com if you've got anything to say to me. There's 4,000 interviews there as well. Send me a message. Other than that, I'll be back again tomorrow with another guest. Speak to you then. Bye for now.