Introducing Flex: Letting Models Write Code (16 min read)

DSPy has separated task definitions from models to allow re-implementing the same tasks in new ways as the AI ecosystem evolves. Initially, BootstrapFewShot selected few-shot examples, followed by MIPROv2 and GEPA improving instructions, and now it leverages the model's coding capabilities to modify the program code itself.

The new module Flex can be integrated into existing Predict, ReAct, and RLM programs. Before optimization, it behaves like a standard Predict module, but when passed to GEPA, it exposes not only instructions but also source code, allowing the reflection model to co-write helper functions, routing logic, and prompts.

Users can assign a cheaper LM for inference and a larger LM for writing code and instructions. GEPA optimizes the Flex program based on given evaluation metrics, and the results are saved via save as readable and comparable source files that can be reloaded.

Optimized programs can handle some inputs via code without model calls, and even when calling models, they first perform parsing and comparison tasks to pass narrower and clearer questions. The goal is to reduce the number of model calls while improving call quality and overall performance.

Code written by the model is treated as untrusted. Flex executes the generated source in a sandbox interpreter, explicitly connecting only the provided tools and predictor calls to the host process, and limits the number of calls allowed in a single execution via max_predictor_calls.

The experiment used the location conflation task, which determines whether two location lists refer to the same actual place. It trained on 1,029 labeled pairs and evaluated on 240 held-out data points, requiring distinction between cases like KIN CAFE that are the same place and cases like CONCESSION #2 KEN MERCER SPORTS PARK that have the same address but are different places.