MIG cuts p95 in half

On Lambda 1x A100 40GB SXM4 with 7 tenants, shared GPU was compared against 7x 1g.5gb MIG slices.

• Model: Qwen/Qwen2.5-1.5B-Instruct
• Server: vLLM OpenAI-compatible server (vllm serve)
• Execution mode: unified as enforce-eager: true for both
• Configuration: Tenant A is the protected target, B~G are noisy neighbors

The key result is clear. Tenant A's quiet-phase p95 was about 2,499ms in shared mode and about 1,319ms in MIG mode. That is a drop of about 1,180ms, or about 47%.

Even during the burst phase when noise increased, Tenant A stayed nearly flat under MIG, while in shared mode it got worse on top of an already elevated latency baseline. In this experiment, the correct interpretation is not simply that bursts were the problem, but that shared co-location itself created a persistent latency penalty.

An important issue also surfaced during reproduction. When vLLM was launched with CUDA_VISIBLE_DEVICES=MIG-<UUID>, device_id_to_physical_device_id() failed while trying to parse the device ID as an integer. In other words, vLLM as of the release at that time could not treat MIG UUIDs as first-class devices. To work around this, the author included a patched version based on vLLM PR #35526 in the bootstrap script.

Things that were tried but excluded from the experiment were also noted.

• A 2-tenant configuration did not generate enough pressure.
• Increasing client load alone did not automatically increase GPU contention.
• Mixing execution modes was inappropriate since it created 2 comparison variables.
• Starting all 7 simultaneously caused KV-cache allocation errors in shared mode, requiring staggered startup.

In conclusion, in this benchmark MIG substantially improved noisy-neighbor isolation. For environments where p95 SLA matters and cannot be disturbed by variability from neighboring workloads, MIG can be a more predictable choice than a shared GPU.