AI-driven memory demand pushes storage to the forefront

As AI agents and large context windows exceed system memory limits, storage is emerging as a core component of AI infrastructure. With GPUs directly generating storage requests to handle thousands of tasks simultaneously, storage systems must also perform data encryption, compression, verification, and recovery.

The Vera BlueField-4 STX, equipped with the NVIDIA Vera CPU, achieved up to 3.21x higher throughput than x86 CPUs in a two-stage compression and encryption pipeline. This enables storage platforms to handle surging AI data throughput with less computing infrastructure.

NVIDIA open-sourced the cuFile API and its underlying vertical storage software stack. cuFile allows GPUs to read and write data directly from storage without passing through the CPU, supporting microsecond-level data access as an open-source component of GPUDirect Storage.

The released APIs are developed in a repository with Google, Intel, NVIDIA, and Meta as initial maintainers, and can be optimized for various hardware and software platforms. NVIDIA explained that this enables faster access to the security contexts and data required for AI-based security defense.

Additionally, NVIDIA is advancing the Storage-Next initiative, involving storage manufacturers, controller, cooling, and orchestration vendors, and standardization bodies. More than 40 storage and flash companies, including DDN, KIOXIA, and Micron, are participating in establishing how GPU-based storage operates and defining open interoperability standards.