Amazon Bedrock Unveils Advanced Prompt Optimization and Model Switching Tool

Amazon Bedrock Advanced Prompt Optimization has been released. Based on a single prompt template, it supports comparing up to 5 models simultaneously, helping either improve the current model or transition to a new one. When switching models, the current model can be set as the baseline with up to 4 additional models added, allowing you to check for regressions in known use cases while refining tasks with lower performance.

Templates must be prepared in JSONL format, with each JSON object written on a single line. You include fixed values like version: bedrock-2026-05-14, along with templateId, promptTemplate, and evaluationSamples, where evaluationSamples contains example user inputs and a referenceResponse. Multimodal variables are placed in inputVariablesMultimodal and specified as PDF or IMAGE, with files either uploaded directly or loaded from Amazon S3. Results are also saved to a specified S3 output location. If needed, you can refine evaluation criteria using customLLMJConfig, evaluationMetricLambdaArn, and steeringCriteria.

Only one evaluation can be selected per prompt template, but multiple templates can be included within a job, allowing different approaches to be mixed.

There are three evaluation methods.

• AWS Lambda: Measures clear metrics like accuracy, F1, and structured JSON matching using custom scoring logic.
• LLM-as-a-Judge: Evaluates responses using user-defined rubrics and scoring scales. The default judge model is Claude Sonnet 4.6.
• Steering criteria: Natural language criteria such as brand tone, format, and safety constraints are provided for the LLM judge to evaluate comprehensively.

Bedrock sends example data and optional correct answers to the inference model, then iteratively rewrites the prompt based on evaluation results to optimize it. The output includes the original and final prompt templates together with evaluation scores, cost estimates, and latency. The feature is available directly from the Advanced Prompt Optimization page in the Bedrock console or via the CreateAdvancedPromptOptimizationJob API, and is currently offered in several regions including Seoul. Billing is based on the Bedrock model inference tokens used for optimization, at the same rate as regular inference.