As businesses continue investing billions in artificial intelligence, a difficult question is emerging. How much of that expensive infrastructure is actually delivering value?
During the 70th IT Press Tour in Silicon Valley, I met with DDN's leadership team, where the conversation centered on a problem that is becoming increasingly expensive for organizations deploying AI. GPUs are getting faster and more powerful, but the infrastructure that supplies them with data often struggles to keep up.
DDN President and Chief Revenue Officer Kevin Delane summed up the company's ambition in a few words:
"Our goal is to make AI profitable."
It sounds like an obvious objective, but with AI spending accelerating and many businesses still searching for a return on their investment, the discussion raised some interesting questions about whether organizations are measuring the right things.
The expensive reality of underused AI infrastructure
One of the biggest problems facing AI infrastructure providers is that GPUs can consume substantial amounts of power while waiting for information. Businesses pay for the hardware, electricity, and infrastructure whether those processors are doing useful work or sitting idle
DDN Chief Product Officer Omer Asad explained that organizations often plan AI data centers around available power, sometimes securing hundreds of megawatts before installing their computing infrastructure. But the systems supplying data can become a performance constraint long before the facility reaches its planned power capacity.
Asad compared the situation to purchasing a Boeing 747 or 777 and leaving it parked. The investment has been made, but the expensive asset is delivering little value. DDN believes the answer is improving how quickly and efficiently data reaches AI processors. The company has ambitious target GPU usage rates approaching 99% in suitable deployments. But actual results depend on workloads, hardware and system configuration.
AI spending is shifting from training to inference.
According to Gartner, spending on AI-optimized infrastructure as a service is forecast to reach approximately $42 billion by the end of 2026, nearly double the previous year. The breakdown is particularly interesting with inference expected to account for $23.3 billion, compared with $19 billion for training.
If these figures prove accurate, 2026 will be the first year inference accounts for the larger share of spending in this category. This shift is worth noting because businesses increasingly need AI systems that can respond quickly, serve many users, and operate at a predictable cost.
An AI customer service platform, for example, might handle thousands of conversations simultaneously. Every delay, repeated calculation, or unnecessary data transfer can increase operating costs. So as AI becomes part of everyday business operations, the cost of generating each response becomes an important commercial consideration.
What NVIDIA and Google reveal about DDN's approach
DDN says NVIDIA has relied on its storage technology since 2016 and currently runs more than 9,000 GPUs internally on DDN infrastructure, including the Eos and Selene supercomputing systems.
Google Cloud is another important partner. DDN's EXAScaler technology powers Google Cloud Managed Luster, a managed file storage service designed for demanding computing workloads.
What makes this relationship interesting is that customers access the service directly through Google Cloud. The underlying DDN technology performs its role without requiring businesses to manage a separate storage deployment.
DDN also shared customer results that offer insight into potential financial benefits. According to its presentation, Salesforce achieved 1.5 times faster model training and 42% lower training costs using Google Cloud Managed Luster powered by EXAScaler.
I should highlight that these are reported customer results rather than guarantees for every deployment. But they illustrate why storage performance deserves attention when businesses calculate AI infrastructure costs.
Faster training can reduce the time expensive computing resources remain occupied, while letting development teams run additional experiments within the same budget.
Why AI inference needs a different approach to data
The move toward inference introduces another infrastructure challenge. AI systems frequently need to recall information they have already processed. This is where a technology called KV cache becomes relevant. Put simply, it lets an AI system retain previously calculated information so it can retrieve it when needed.
DDN has developed Infinia, a data platform designed to support modern AI applications, including inference workloads. It works alongside EXAScaler, which remains focused on demanding training environments and also integrates with NVIDIA's Dynamo software. What makes this valuable to businesses is moving cached information directly between storage and GPU memory, reducing delays associated with conventional data access methods.
DDN claims this approach can deliver 4.2 times faster time to first token in a test involving 100,000 tokens of context. Time to first token measures how quickly an AI application begins responding after receiving a request.
For businesses building AI applications, that translates into something customers understand immediately: how long they wait before the system starts answering.
The commercial implications extend to operating costs. If an application can retrieve previously processed information rather than recalculating it, it can serve additional requests using the same computing capacity. The benefits will depend on how often information can be reused, the nature of each workload, and the cost of maintaining the additional cache infrastructure.
Why performance at scale changes the business case
I came away from DDN thinking that the conversation around AI infrastructure needs to become much more focused on financial outcomes.
Our newsfeeds are filled with leaders discussing how many GPUs they own, how much computing power they can access, and how sophisticated their models have become. But these measurements are more of a pissing contest and only tell only part of the story.
Businesses also need to understand how long training jobs take, what each AI response costs, how much energy is consumed, and whether their existing infrastructure can support additional demand.
DDN's experience with NVIDIA and Google shows it can operate in demanding environments. At the same time, its newer inference technology reflects where the company believes commercial AI adoption is heading.
The challenge for customers will be establishing which improvements translate into measurable savings in their own environments.
With so much money flowing into AI infrastructure, businesses should ask how effectively they're using their technology investments and what changes would improve returns.
After hearing DDN's perspective in Silicon Valley, I suspect the next phase of enterprise AI investment will involve much closer scrutiny of the cost of every training run, every response, and every hour of GPU computing time.
I will be speaking with the DDN on my podcast. If you have any questions you would like me to ask on your behalf, please let me know, and I will bring them into the conversation.
Special thanks to Alex Zaharov-Reutt, iTWire's Global AI and Technology Editor, for filming the presentation and allowing me to use the footage.