Ensuring LLM Reliability Through Evaluation Driven Development (EDD)

When operating generative AI-based products in production environments, the biggest challenge is ensuring Reliability. Due to the nature of LLMs, which are based on probabilistic functions, even minor prompt modifications frequently cause Regression, where performance improves in certain areas while degrading in others.

Dosu, an AI software development team, adopted the Evaluation Driven Development (EDD) approach to solve this problem. This is a concept similar to Test Driven Development (TDD), where 'Evals' serve as the standard for measuring whether an update maintains existing performance while achieving the targeted improvement.

The EDD workflow practiced by Dosu is as follows:

• Define new behavior and create initial Evals
• Release the new feature to users
• Identify failure modes through production environment monitoring
• Add identified failure modes to offline Evals
• Iteratively update evaluation metrics to improve performance
• Re-release and repeat based on improved metrics

Initially, they relied on a manual approach of directly reviewing logs, but as the service scaled, they have maintained high quality even at large scale by automating monitoring and search using LangSmith.