Unified API for AI Model Routing

Google Cloud API Gateway offers AI model routing in Public Preview. Developers can send OpenAI-compatible requests to Gemini, Claude, and OpenAI OSS-GPT without hardcoding endpoints or managing open-source proxies themselves.

API Gateway acts as a serverless ingress layer, supporting simple rate limiting and token tracking when used standalone. When integrated with the Gemini Enterprise Agent Platform, the Agent Gateway manages agents' external requests via security policies, after which API Gateway can dynamically route them to Google-hosted LLMs.

Routing configuration consists of the following steps:

• Define backends and virtual model names in the x-google-api-management extension block of the OpenAPI 3.x specification.
• Use x-google-model-router to specify routing rules between primary and fallback models.
• Deploy the updated API configuration.
• Applications send standard OpenAI POST requests, and the Gateway translates them into the backends' native schemas before forwarding.

All backends referenced by a single router must use the same host. Routing works by changing the model and path on the shared Vertex AI host; traffic routing between different hosts is not supported.