The most expensive component in an AI system can also sit idle. Many are investing heavily in GPUs to support increasingly demanding AI workloads, only to learn the hard way that processors can only work as quickly as the data reaches them.
Welcome to the infamous, expensive problem experienced by those still struggling to understand the true cost of their AI investments.
During the 70th IT Press Tour in Silicon Valley, I visited ScaleFlux at its Milpitas headquarters, where CEO Hao Zhong and his team offered an interesting perspective on the infrastructure conversation. While much of the industry's attention remains focused on processing power, ScaleFlux believes memory and storage are becoming major constraints on AI performance.
The company has spent the past 12 years developing computational storage technology and the controllers that manage how information moves through computing systems. Its latest direction brings together NVMe storage, CXL memory technology, silicon, and firmware to address the fact that AI infrastructure is becoming increasingly powerful. Still, the systems that supply data are struggling to keep pace.
The GPU problem that businesses rarely hear about
One of the most interesting points raised during the presentation concerned how much time expensive GPUs can spend waiting for information. ScaleFlux illustrated the substantial bandwidth differences between GPUs, DDR5 memory, and SSD storage, showing why data movement can become a constraint as AI systems grow.
The company's figures suggest that DDR5 bandwidth can be approximately 37 to 40 times slower than the GPU bandwidth in its comparison, while PCIe 6.0 SSD bandwidth can be hundreds of times slower. These measurements describe different technologies with different roles, but the wider point is worth understanding. Processor performance alone tells businesses little about the productivity of the complete system.
Why memory is becoming an expensive headache
ScaleFlux also highlighted another industry problem: demand for memory and storage is growing at a remarkable rate. The company's illustrative estimates for 2027 suggest AI and enterprise systems could require between 500 and 600 exabytes of NAND flash storage, alongside 20 to 30 exabytes of DRAM.
This is where ScaleFlux's background in computational storage lets them create real value. How? The technology lets selected processing tasks run within or closer to storage, potentially reducing the amount of information transferred to the CPU or GPU. Its transparent compression technology also increases effective storage capacity by reducing the physical space needed for suitable data.
Could CXL change how businesses use memory?
Another technology discussed extensively during the visit was Compute Express Link, or CXL. For readers unfamiliar with the technology, CXL provides a high-speed connection that allows compatible processors and systems to access additional memory resources. It can help overcome some traditional limitations of installing memory directly inside individual servers.
ScaleFlux sees an opportunity to introduce another memory tier between conventional system memory and SSD storage. This could give data center operators greater flexibility in allocating memory to different workloads, particularly where capacity requirements vary, or expensive memory resources are underused.
The company's MC600 controller supports CXL 3.2 and PCIe 6.0, with both DDR4 and DDR5 compatibility and support for configurations of up to 2TB. ScaleFlux says the controller is designed for enterprise-grade reliability, security, and power efficiency. During the October presentation, it was still in sampling with selected customers.
Building SSDs around the application
Perhaps the most interesting part of the ScaleFlux presentation was how the company wants to change how storage products are developed and delivered.
Traditionally, businesses have purchased SSDs from established manufacturers. Product decisions were typically made based on performance, capacity, and price. But ScaleFlux argues that the growing variety of AI workloads makes this approach increasingly restrictive.
The problem is that AI training, inference, cloud infrastructure, and enterprise applications have different requirements, with each placing different storage demands. ScaleFlux believes the answer is a Lego-like approach that enables teams to assemble storage products using ScaleFlux controllers, firmware, and reference designs.
Best of all, customers can select compatible NAND and DRAM components from different suppliers and choose manufacturing partners that meet their commercial requirements.
Its FC6116 PCIe 6.0 SSD controller demonstrates the direction. ScaleFlux claims controller capabilities of up to 28GB/s bandwidth and seven million IOPS, with support for SSD capacities ranging from enterprise boot drives to 256TB. The company was sampling the controller during our visit, with wider availability dependent on customer qualification.
What does this mean for businesses investing in AI?
I came away from ScaleFlux with a better appreciation of how much AI performance depends on the technology surrounding the GPU. It is easy to understand why processors attract so much attention. They are expensive, their capabilities are improving rapidly, and the companies developing them dominate many AI investment conversations.
Memory capacity, storage performance, data movement, and power consumption all influence how effectively those processors can operate. This is why many businesses still see disappointing results when their applications cannot access information quickly enough, despite investing heavily in additional computing resources.
ScaleFlux's approach is interesting because it addresses several of these problems at the silicon and firmware level. However, businesses should distinguish between the performance capabilities demonstrated by controllers, technologies already deployed in production, and products that remain in customer testing.
If businesses want better returns from their AI investments, understanding how information moves through their systems seems like a sensible place to start.
I will be speaking with the ScaleFlux team on my podcast. If you have any questions you would like me to ask on your behalf, please let me know, and I will bring them into the conversation.
Special thanks to Alex Zaharov-Reutt, iTWire's Global AI and Technology Editor, for filming the presentation and allowing me to use the footage.