Graphsignal releases graphsignal-top, a live GPU monitoring tool for inference engines

Graphsignal has released graphsignal-top, a command-line interface for its open-source profiler that provides real-time visibility into GPU inference performance. Designed to replace manual JSON inspection, the tool offers a top-like experience for monitoring vLLM, SGLang, and other inference engines wrapped with graphsignal-run.

Key Features

The interface displays several critical metrics in real time:

• GPU Health: Utilization over time, memory usage, power consumption, and throttling status.
• Engine Metrics: Running/waiting request counts, tokens per second (tok/s), and Time to First Token (TTFT) / Time Per Output Token (TPOT) at p50 and p95 percentiles.
• GPU Time Split: Breakdown of time spent on kernels, graphs, memcpy, and synchronization.
• Kernel Ranking: Every kernel is ranked by execution time over the last 5 seconds, displaying readable names and categories instead of mangled symbols.

Practical Example

In a demonstration using Qwen3-32B on 2x RTX PRO 6000 GPUs with tensor parallelism (tp 2) over PCIe, the tool revealed that NCCL all-reduce accounted for a quarter of GPU time, while half the wall time was spent on host synchronization waiting for it. This level of detail allows developers to identify bottlenecks like communication overhead at a glance.

Remote Monitoring

The tool supports remote connections via graphsignal-top --host gpu-box against a profiler started with --listen-host, or through SSH port forwarding. A --once flag allows for single-frame output suitable for logging.