Resolving Two Blind Spots in Amazon Bedrock LLM Gateways: Caller Identification and Model Governance

Introducing an LLM gateway in front of Amazon Bedrock offers management convenience, but creates a blind spot where all requests are processed under the gateway's single IAM role, causing the actual caller's identity to be lost. This results in the inability to perform caller audit trails and obscures model governance, which involves monitoring usage and performance per model.

To resolve this, the gateway should remain lightweight while leveraging Amazon Bedrock's native features to secure governance across two axes.

1. Caller Axis (Who, which team made the call)

• Problem: Bedrock logs only retain the gateway role, making it impossible to identify the calling entity.
• Solution: Use the X-Amzn-Bedrock-Request-Metadata header to inject metadata such as user, team, and call identifiers.
• Result: Information is recorded in the requestMetadata field of the Model Invocation Log, allowing the actual caller to be restored and audited using Bedrock logs alone.

2. Model Axis (Which models are used and how much)

• Problem: Opaque profile ARNs make it difficult to track usage and control access per model.
• Solution: Create Application Inference Profiles and attach tags to manage them by model.
• Result: Model-level metrics can be observed via CloudWatch, and access can be controlled by restricting IAM permissions to the profile ARN.

This architecture is based on Claude Code and LiteLLM, using a pass-through method where the gateway forwards requests without transformation, thereby maintaining Claude's latest features (such as Prompt Caching) while enhancing security and audit capabilities.