Multi-Agent Kernel Optimization

Cursor, together with NVIDIA, performed CUDA kernel optimization using a multi-agent harness. Over 3 weeks, it autonomously handled 235 problems, optimizing Blackwell GPU kernels from scratch down to the assembly level, achieving a 38% geomean speedup.

Problem generation and evaluation were carried out with SOL-ExecBench. This benchmark creates problems based on real inference and training workloads drawn from 124+ production open-source models, and measures performance on 27 NVIDIA Blackwell 200 GPUs. Cases where results were gamed beyond hardware limits—via caching or similar tricks—were invalidated during the verification stage.

The experiment was run using a coordination protocol defined in a single markdown file. A planner agent distributed and rebalanced work among autonomous workers, and the system called the benchmarking pipeline directly during execution, iterating through testing, debugging, and optimization. Solutions were written separately in two languages.

• CUDA C with inline PTX: verifies low-level hardware reasoning ability by working directly down to registers and the ISA level
• CuTe DSL: verifies the ability to learn high-level abstractions, which are underrepresented in public training data, from documentation alone

Results were presented in two ways. The baseline was single-agent-optimized PyTorch code, and performance was measured using Geomean speedup and SOL score, which represents performance relative to the theoretical limit. The multi-agent system beat the baseline in 149 out of 235 (63%) cases, and achieved a 2x or greater improvement in 45 cases (19%).

Representative examples were also presented.

• BF16 Grouped Query Attention with Paged Prefill: optimized the prompt-phase attention for SGLang's Llama 3.1 8B in CUDA C++, achieving an SOL of 0.9722 and an 84% geomean speedup over baseline; after swapping it into SGLang, a 3% TTFT improvement was confirmed
• NVFP4 MoE Linear with Gating: identified a quantization bottleneck, fused the scale calculation and rounding, and exploited the fact that NVFP4 has only 16 possible values by mapping directly to threshold buckets, achieving a 39% geomean speedup and 0.58 SOL
• BF16 Matrix Multiplication: learned Blackwell-specific instructions and memory access patterns to generate a custom GEMM kernel, reaching 86% of the NVIDIA cuBLAS human baseline, and was up to 9% faster on small-M tests

Cursor notes there is still significant room for improvement, with a median SOL of only 0.56. Still, this experiment demonstrates that the multi-agent architecture is effective for tackling long-horizon, open-ended software problems, and signals that these techniques will be reflected in Cursor's core product going forward.