Cloudflare launches Pay Per Use and Monetization Gateway to charge AI agents

Cloudflare reports that AI agent traffic on its network grew by more than 1,700% over the past year, with non-human traffic now accounting for more than half of all Internet requests. This shift has broken the traditional web economy, where search engine referrals drove ad revenue, as AI answer engines consume bandwidth without sending humans to sites. In response, Cloudflare is introducing new tools to help site owners identify, control, and monetize this new audience.

Identifying and Controlling Agents

To distinguish between beneficial agents and extractive crawlers, Cloudflare launched Web Bot Auth, a cryptographic signing mechanism used by operators like OpenAI, Google, and AWS to verify agent identity. The company also replaced its single "block AI bots" switch with separate controls for Search, Agent, and Training traffic. Additionally, the new Disallow AI Training feature allows sites to remain indexed for search while instructing crawlers not to use their data for model training, a commitment honored by Apple, Google, and Microsoft.

New Monetization Models

Cloudflare is launching two beta products to help content creators and service providers capture value from AI agents:

• Pay Per Use: A licensing bridge for publishers who cannot negotiate bespoke deals with every AI company. It pays publishers when their content is actually used by verified buyers, providing transparency into which questions surfaced their work.
• Monetization Gateway: A tool for API providers and services to charge agents per request, query, or token. It uses the open x402 protocol to return an HTTP 402 Payment Required status, allowing agents to pay sellers directly for access.

Infrastructure and Standards

These tools are built on open standards to prevent a few companies from controlling the agentic web. Cloudflare also announced a joint research project with OpenAI to improve how AI search engines discover and index content efficiently. To reduce wasted bandwidth, the company is introducing Markdown for Agents and WebMCP, which allow agents to read pages and interact with services more efficiently than traditional HTML scraping.