AI BriefingAI Gateway begins supporting Service Tiers feature
Key point
AI Gateway has introduced a Service Tiers feature that lets users optimize latency, throughput, and cost according to their use case.
Details
AI Gateway has started supporting a Service Tiers feature that lets users optimize performance and cost to match their workload characteristics. Users can freely choose from a fast tier for conversational workloads to a low-cost tier for background tasks.
Service Tiers are currently available for OpenAI and Gemini models, and can be used through all AI Gateway API formats, including the AI SDK, Chat Completions API, and Anthropic Messages API. Billing is automatically adjusted based on the tier used for each request.
The available tiers are divided into three types as follows:
- default: Standard processing method
- priority: Higher cost but faster processing speed
- flex: Lower cost but may incur latency
Users can easily apply this by setting serviceTier within providerOptions.gateway. When a specific model is served by multiple providers, it's also possible to set a separate tier for a specific provider only (e.g., Vertex AI). Service Tiers operate on a best-effort basis, and if application is not possible, the request runs on the default tier.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.