Agent Observability: How to Monitor and Evaluate LLM Agents in Production

Traditional software has fixed code paths and limited input values, so APM (Application Performance Monitoring) metrics like error rates or response times are sufficient for monitoring. However, LLM agents use natural language as input, giving them an infinite input space, and they have non-deterministic characteristics where results can vary even with the same input.

Agents go beyond simple code paths to perform multi-step reasoning, Tool Call, and Retrieval operations, making it difficult to predict all behavior during development. Therefore, agent observability in production requires, beyond system metrics, the ability to monitor Conversations themselves.

Successful agent operation requires the following differentiated approach.

• Addressing uncertainty in the input space: Since it's impossible to predict how users will phrase their questions, the ability to identify various intents must be monitored.
• Managing prompt sensitivity: Since results can change even with minor wording variations, consistency of behavior between development and production environments must be continuously verified.
• Conversation-centric monitoring: Beyond system resources, it is essential to track the quality of text generated by the agent and its reasoning process.