KernelEvolve: How Meta's Ranking Engineer Agent Optimizes AI Infrastructure

Meta runs large-scale AI services on heterogeneous hardware mixing NVIDIA GPUs, AMD GPUs, MTIA chips, and CPUs. As the number of models and chip generations grows, the number of kernels explodes, and manual tuning by humans no longer scales.

To address this, Meta introduced KernelEvolve. This agent treats kernel optimization not as a single code generation problem but as a search problem, running an automated loop that generates, evaluates, and analyzes hundreds of candidate implementations.

The results are clear. NVIDIA GPU inference throughput for the Andromeda Ads model improved by over 60%, and training throughput for the MTIA-based ads model improved by over 25%. This process compresses work that would take a skilled engineer weeks into hours of automated search.

The core architecture consists of four parts.

• LLM Synthesizer: Generates candidate kernels suited to multiple languages and targets, including Triton, CuTe DSL, FlyDSL, CUDA, HIP, and MTIA C++.
• Tree Search Engine: Explores using Monte Carlo tree search and evolutionary strategy, selectively leveraging context from parent and sibling nodes.
• Retrieval-Augmented Knowledge Base: Retrieves correctness constraints, platform-independent optimization guidelines, and hardware-specific documentation to inject into prompts.
• Job Harness: Handles compilation, correctness verification, performance measurement, profiling, and failure recovery, withstanding long build times and unstable environments.

This approach makes it possible to keep automating kernel optimization even as new model architectures or new chip generations emerge. Through this, Meta points toward a direction of accelerating model development speed and more quickly securing production-ready kernels across heterogeneous AI hardware.