GraphSignal GPU Profiler for AI Agents Released

Existing GPU profiling tools rely on humans visually inspecting timelines, limiting the ability of AI agents to autonomously tune performance. To address this, GraphSignal, an open-source profiler designed to allow AI agents to directly read and analyze measurement data, has been released.

Key Features and Characteristics

• Agent-friendly data provision: By wrapping various inference engines such as vLLM, SGLang, and llama.cpp, or general CUDA/ROCm processes in a wrapper, all measurement data is provided via a single JSON endpoint (/signals).
• Comprehensive metric collection: Integrates kernel execution times, CUDA graphs, memory copy and synchronization times, NVML data (utilization, VRAM, power, clocks, throttling, XID errors), Prometheus metrics, and tracebacks from console output.
• Non-invasive application: Usable without code modifications or imports, runs locally, and does not upload data if no API key is present.
• CUDA graph debugging support: The --cuda-graph-trace node option allows kernels within CUDA graphs to be separated by individual names for detailed analysis.

How to Use

Developers start profiling by wrapping existing execution commands with the graphsignal-run command. Then, by providing the repository's SKILL.md to an AI agent and instructing it on a goal (e.g., "Identify and fix the cause of 40% GPU utilization during decoding"), the agent autonomously performs a loop of execution-analysis-configuration change-re-execution. It is also possible to manually inspect data using curl and jq.