4B Parameter RL Model Reduces Postgres Query Planning Latency by 44.7%

Rohan Bansal released a 4B parameter model that achieves an average 44.7% reduction in latency and a geometric mean speedup of 1.81x compared to Postgres's default query plan generation. The model was developed to address the NP-hard limitations noted in the Leis et al. study and Postgres's cardinality estimation errors.

Training Methodology

The model was trained by combining off-policy distillation and reinforcement learning. It solved the qo-agent harness language acquisition problem through trajectory-based off-policy distillation from GPT-6 Astra, and designed a custom GRPO variant to score RL rollouts in noisy environments.

Performance Benchmark Results

Performance was evaluated using the IMDb-based JOB benchmark (113 queries). After 1,200 updates of RL training, the following results were achieved:

• Geometric mean speedup (Sgeo): 1.41x
• Total workload speedup (Sworkload): 1.29x
• Valid trajectories: 101/113
• 3-rollout best-of-15 method: Achieved an exponential mean speedup of 1.81x

In Frontier model validation, GPT-6 Astra recorded an Sgeo of 2.54x with 5 candidates, but the 4B model demonstrated competitive performance even with limited resources.

Infrastructure

Training was distributed using 2x H100 nodes and Postgres containers on desktops, with vLLM inference and the trainer operated separately. The code is publicly available on GitHub (polyphilz/qorl).