OpenAI Engineer Shares Case Study: Agent Speed Up 50% and Cost Down 30% via Prompt Refactoring

Wulfie, an engineer at OpenAI, argues that in AI agent development, prompts should be treated as Products and Code, not just simple text. Extending prompts lengthily with each feature addition, as in traditional methods, accumulates contradictions and ambiguities, forming 'spaghetti prompts' that directly lead to agent errors.

Core Principles of Prompt Management

To improve prompt quality, principles from software engineering should be adopted. The author recommends applying the MECE (Mutually Exclusive, Collectively Exhaustive) principle to eliminate redundancy between sections and prevent omissions. Additionally, separating Backend (tool usage, logic) from Frontend (output format, tone) enhances maintainability, and clearly defining conditional logic reduces reliance on the LLM's implicit knowledge.

Measured Results and Implications

Cases have been confirmed where actual business metrics improved through such structuring and refactoring. After prompt rewriting, agent speed improved by 50%, 7-day retention increased by 40%, and costs were reduced by 30%. In the case of Loveable engineers, eliminating prompt inconsistencies resulted in annual LLM cost savings of $20 million.

Prompt optimization makes it easier to remove unnecessary instructions as model performance improves, so adopting a structured design from the beginning is advantageous in the long run. Clear human specification of preferences remains key rather than full automation, requiring an approach that manages prompts precisely like product specifications.