Amazon Launches 'CloudWatch Omni' GA for Observability of Generative AI and Agent Workloads

Amazon has launched CloudWatch Omni as GA (Generally Available), an integrated observability solution designed to address monitoring limitations caused by the non-deterministic nature of generative AI and agent workloads. The solution supports the design, evaluation, and operation of AI agents regardless of model provider, framework, or runtime, focusing on detecting response quality degradation and regression issues that are difficult to identify with traditional metrics.

Key Features and Usability

CloudWatch Omni provides two usage surfaces for developers and operators. Developers can inspect traces and run evaluators locally via native extensions for VS Code and Kiro, while operators monitor fleets through a standalone web experience separate from the AWS Console, using SSO authentication. Both surfaces share the same data, and local development is possible without a cloud connection.

Key features include capturing all traces, generating quality scores via 17 built-in evaluators (such as coherence, helpfulness, and faithfulness), comparing prompt versions in the Playground, generating test datasets based on production traffic, and automatic regression detection. In particular, Trace Explorer displays LLM calls, tool calls, and reasoning steps in a hierarchical timeline to pinpoint the causes of malfunctions, while Ask Assistant uses AI to analyze traces, detect anomalies, and answer questions.

Integration and Compatibility

CloudWatch Omni integrates with AI code assistants such as Kiro, Claude Code, and Codex to automate Dev Server setup, dependency installation, and instrumentation. This allows users to complete the process from installation to running the first traced agent session within minutes. Supported frameworks include LangChain, LangGraph, CrewAI, OpenAI SDK, Strands, and Vercel AI SDK, with support for Python and TypeScript.

In terms of standards compliance, it uses the OpenInference and ADOT open standards and can run in various environments such as Lambda, ECS, and EKS. It also supports integration with third-party evaluators like Braintrust, DeepEval, and Ragas, allowing users to maintain existing workflows without replatforming. Amazon Bedrock AgentCore supports native observability, and AgentCore evaluation features can be utilized within the Omni workflow.

Pricing and Availability

CloudWatch Omni is currently GA, and IDE extensions are provided for free. Users can explore features locally without an AWS account, but AWS credentials are required to use Amazon Bedrock models, and API keys are needed for other providers such as OpenAI or Anthropic. API calls, documentation search, and troubleshooting support are also available via the AWS MCP Server and plugins.