Cloudflare's AI Platform: an inference layer for agents

Cloudflare announced that it is combining AI Gateway and Workers AI to let developers use models from multiple providers as a single unified inference layer.

• With a single AI.run() in Workers, you can call third-party models from OpenAI, Anthropic, and others.
• A REST API is coming soon, making the same model catalog available outside of Workers as well.
• It currently offers 70+ models and 12+ providers through one API and one credit system.

From an operations standpoint, it strengthens cost visibility and observability. You can attach custom metadata to requests to break down AI spend by team, user, or workflow. The message is that centralized cost management is essential in environments that mix multiple models and providers.

Cloudflare also outlined a Bring Your Own Model direction. Using Replicate's Cog, you'll be able to package model dependencies and inference code, then deploy it to Workers AI to serve your own models. Looking ahead, it also previewed customer-facing APIs, wrangler commands, and faster cold starts based on GPU snapshotting.

As a key performance point for agents, it emphasized time to first token. Cloudflare leverages its edge network spanning 330 cities worldwide to reduce network latency between users and inference endpoints, and Cloudflare-hosted models run within the same global network with no additional public Internet hop.

On the reliability side, it offers automatic failover and stream recovery features. With multi-provider models, if one provider fails, requests are automatically routed to another provider; and if a connection drops during a long-running agent task, AI Gateway buffers the stream so it can resume after reconnection. Combined with the Agents SDK's checkpointing, this reduces the perceived impact of interruptions.

Finally, the Replicate team has joined Cloudflare's AI Platform team, and integration work is underway to bring Replicate models into AI Gateway and re-platform hosted models onto Cloudflare's infrastructure.