Open-source LLMOps platform agenta released

agenta is an open-source platform that integrates prompt management, LLM evaluation, and LLM observability for developing production-grade LLM applications.

Key Features:

• Prompt Management: A UI-based playground lets you compare and experiment with multiple prompts alongside test cases, and supports version control through branch and environment concepts.
• LLM Evaluation: Verifies quality using an LLM-as-judge approach with 20+ pre-built evaluators and custom evaluators, and supports integrating expert feedback (Annotation).
• LLM Observability: OpenTelemetry-based tracing tracks cost, latency, and usage patterns, and lets you debug complex workflows.

Users can use the managed service via Agenta Cloud, or self-host it directly on their own server using Docker Compose.