Why the Future of AI Belongs to Data Infrastructure, Not Bigger Models

The story of AI has traditionally been told in numbers. More data, more parameters, more power. But behind the rapid scaling of models, a quieter transformation has been taking place in the foundations that make those systems useful. 

Beneath every large language model lies a layer of infrastructure responsible for how information is remembered, retrieved, and understood. That is where companies like Pinecone are reshaping the conversation. 

Pinecone founder and chief scientist, Edo Liberty, shares why he believes the next leap in AI will not come from building bigger models but from improving how those models think with data.

Key Takeaways

• Vector databases form the foundation of modern AI knowledge systems.

• Basic RAG delivers 80% accuracy without complex fine-tuning.

• Context engineering is emerging as a vital AI discipline.

• Pinecone's architecture strikes a balance between cost, speed, and scale, making it suitable for real workloads.

• AI's future depends on merging data infrastructure with model intelligence.

• Vector databases will soon "know" information like a well-trained human expert.

Making AI knowledgeable

In our conversation, Liberty laughed about the industry confusion around his company's technology. "We called it a vector database when we came out. Nobody knew what a vector database was. We started selling our service. We knew how to work it because I had experience working at Amazon, Yahoo, and other places. We built vector search technologies, which were then referred to as machine learning. So we knew exactly what it was. But even after we launched the service, we didn't know what it was."

Liberty compared this to the human mind's ability to recognize faces in a crowd. "Your brain somehow did like a database operation. That's exactly the kind of stuff that happens inside a vector database. It's a search machine, a very elaborate mechanism that allows you to really represent and access data in a way that it's represented numerically by those networks."

How RAG Makes AI Systems More Useful With Less Effort

Retrieval-augmented generation, or RAG, has become one of the most popular methods for giving AI systems knowledge about specific topics or companies. Liberty argued that the simplest versions already outperform many fine-tuning approaches.

He explained why. "What people even try to do, RAG or fine tuning for, they have a very natural and standard problem, which is, okay, these models are compelling and have a lot of skills, and they seem to be incredibly intelligent, but they know absolutely nothing about my business, my customers, my meetings, my products, my what have you. And for them to be effective in me building agents, in me building capabilities in my company, they have to know something about my company."

Fine-tuning once looked like the answer, but proved unreliable. "It ended up not being a thing for multiple reasons. It's tough to do in general. Most companies lack the talent to do that. But more importantly, it's nearly impossible to do well because you take a final model, tweak it a little, and then you have no idea what happened. You might have made it slightly better at understanding your product, but now you might have given it a frontal lobotomy inadvertently, and now it doesn't know how to read anymore." But RAG is simpler and safer.

Context is Everything

During our talk, Liberty described how the quality and structure of context can change a model's output. "Providing the right context to the LLM is not one thing. For example, we know now from fairly well understood and replicated research that providing more context is actually making things worse."

Placement and sequencing, he explained, matter far more than many realize. "If it's in the middle, it's going to perform worse. If it's in the beginning or the end, it's actually going to perform better. The length matters. The sequencing matters. Even the structure matters."

That led him to a new phrase that captures an emerging discipline. "There is almost like a mini specialization now called context engineering. Some people know how to, you know, sort of like massage things to make things better for different models and so on."

Liberty's team is focused on combining what models know with how they remember. "For us to be able to combine what machines know and how they remember everything and how they think and what skills they have, those are separate. We are squarely focused on knowing, remembering, and fetching the right insights at the right time, and working very closely with the model manufacturers so that we can now combine those two things and provide great outcomes."

Balancing Performance and Cost

Behind every successful AI product lies a deep infrastructure challenge. Liberty described how Pinecone has evolved to handle both high-volume and infrequent workloads at scale. 

"The access patterns of data and the access patterns for vectors are very different than the access patterns of tables in a data lake. For us, this is very much a moving target. The market is evolving so quickly that we ourselves are learning."

He compared these patterns to how memory works. "If you think about agents, if they need to go now and complete a task, they might now require access to information that nobody has touched in hours or days. But then they're going to run 500 queries to get every part of it, dissect it, and retrieve the information. And they need to do this incredibly rapidly."

The company recently re-architected its system to meet these demands. 

"We made all of our storage based on what's called object storage, very cheap, durable, long-term storage. We augmented the vector databases with indices, essentially adding a vector next to your raw data. And there is a very sophisticated caching layer that loads just the right information at the right time to be able to compute the top results."

Performance, he said, depends on separating how data is written from how it is read. "When you update things, when you write things, you really can't mess with those cache machines, those machines that actually serve queries because they might require, you know, they might need to do this a thousand times a second, and any disruption and any delay is actually causing a real issue."

The result is a system designed to handle what Liberty called the "balance of performance, scale, and cost." For Pinecone's 5,000 customers, that equilibrium determines whether AI applications remain fast and affordable enough to reach production.

The Convergence of AI and Data

Liberty described the present moment as the merging of two enormous human efforts. "There are two completely different sides to this. One of them would be like a mega cycle in technology, and now they're happening together because of this collision of worlds."

He traced how the industry reached this point. "We think about AI really as a 30-, 40-, 50-year journey from like PAC learning and theories about single neurons and all that stuff and how we've got to where we are today. We had to invent GPUs and algorithms and data sets and architectures for networks and all that stuff."

Then he widened the lens. "Knowledge and data are even bigger. You have, with you know, created the internet, for crying out loud, like all the open encyclopedias, Wikipedia, ontologies, databases, and so on. It's just to organize data. What you have is literally millions of human ingenuity years, like engineering years, on one side and another side colliding."

That collision, he said, is what makes the current era so transformative. "Suddenly, you can infuse these brilliant models with information because now you can bring data and search to help AI be better. So now you can eliminate hallucinations. Now you can make them really do the work of a doctor or a lawyer."

A Glimpse of What Comes Next

Looking ahead, Liberty predicts that vector databases will evolve from storage systems into engines of understanding. "You're going to expect your vector database, or your context engine, or your model, not your model, well, it's not the right, but your system that deals with knowledge, to be able to consume huge amounts of information and just know it. Know it as if a brilliant, capable person sat there and read through all of it and figured it out and made connections."

That shift, he said, mirrors the progress of the past few years in modeling. "That ability to consume information and know it is still today where models were maybe in 2020."

For Liberty, making AI knowledgeable has never been about hype or headlines. It is about building the infrastructure that enables lasting intelligence.

As AI systems continue to grow in size, Liberty is reminding the industry that accurate intelligence begins with structure, not scale. "We're just incredibly fortunate and thankful that we can sit in the middle and be where these two giant waves are colliding."

The Bottom Line

The next phase of AI progress will hinge on data infrastructure, not model scale. Edo Liberty's insights reveal that the intelligence of tomorrow's systems will depend on how they remember, retrieve, and connect information. 

Vector databases are emerging as the hidden force transforming raw data into usable knowledge, bridging the gap between what machines process and what they genuinely comprehend. The organizations that focus on strengthening this foundation will set the pace for a more thoughtful and capable generation of AI.