AI BriefingGPU-Usage-Audit: What if your GPU isn't just idle, but idle while 'holding onto memory'?!
Key point
An open-source tool that tracks resource waste by distinguishing between actual GPU compute time and memory occupancy status.
Details
Even when utilization (util) shown in nvidia-smi is low, if a Jupyter notebook or similar process is holding GPU memory, other users can't use that resource. This leads to serious cost waste in shared servers or GPU environments billed by the hour.
GPU-Usage-Audit is a tool that logs GPU metrics to SQLite via the NVIDIA driver and provides detailed reports. Unlike existing tools, it precisely breaks down GPU usage time into three states for analysis.
- Actual compute time: Time the GPU is actually performing computation
- Fully idle state: Time the GPU is completely empty
- Memory-held idle state (idle-held): Time when memory is occupied but no computation is being performed
This tool converts 'held time' into GPU-hours for display, allowing you to see occupancy status by user. It can be easily installed and run with the command uv tool install gpu-usage-audit.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
NVIDIA Open-Sources Local AI Inference Router 'PAIR'NVIDIA Releases Open-Source LLM Router 'Switchyard'… Up to 30% Cost Savings Compared to Opus