ModelExpress: Deploying Model Artifacts at the Speed of Light

As model checkpoints grow from hundreds of GB to TB-scale, the cost of moving model weights within a cluster is skyrocketing. Across every workflow—cold starts, autoscaling, RL post-training—moving weights acts as a mandatory 'time tax' that must be paid before any valid work can begin.

NVIDIA ModelExpress (MX) is built on a simple idea: before loading a model, first check where a compatible copy of the weights already exists. Instead of treating every replica as an independent cold start, it selects the fastest available source and transfer path.

Here's how it works:

• P2P RDMA transfer: If a serving peer already holds compatible weights on its GPU, MX transfers them directly from GPU to GPU via the NVIDIA Inference Xfer Library (NIXL). This eliminates redundant access to object storage, local disk, and host memory.
• Storage bootstrap: If no peer is available, MX bootstraps via the fastest available path—streaming directly from object storage—skipping the need to save to disk or read local files directly.

In real-world performance measurements, MX transferred DeepSeek-V4 Pro weights and JIT kernel cache artifacts from a serving replica to a new replica in under 10 seconds. This delivered a remarkable result, cutting total startup time from 8 minutes down to 1 minute 44 seconds.