Cloudflare x402, Web Bot Auth, and Pay Per Use: What Birthday Week 2026 Changes for AI Agents

Cloudflare used its 2026 Birthday Week (September 28 to October 2) to describe AI agents as the web's "second audience" and to launch new ways for sites to charge that audience: x402 payments over HTTP 402, and Pay Per Use. Daily requests from AI agents on Cloudflare's network grew more than 1,700% in a year, by the company's count. The new launches sit on top of tools Cloudflare already had for seeing which bot is visiting (Web Bot Auth) and deciding what it may do. This post separates what is new from what was already shipping, and covers what changes for teams whose agents or pipelines read public pages.

Key Takeaways

• Cloudflare reports that daily AI agent requests on its network grew more than 1,700% over the past year, and that AI training rose from 22% to 52% of crawler requests between spring 2025 and June 2026, by crawlers' stated purpose (Cloudflare, 2026).
• New in Birthday Week: Pay Per Use (beta), the Monetization Gateway (closed beta for U.S. sellers and buyers), and a Web Search API in AI Gateway. They build on tools Cloudflare already had: AI Crawl Control, Web Bot Auth (cryptographic bot identity), and the separate Search, Agent, and Training controls it introduced in July.
• The Monetization Gateway charges agents with HTTP 402 over the x402 protocol, settling in USDC on Base (Cloudflare, 2026).
• For agent builders, the practical shift is from "allowed or blocked" to "identified, priced, or open." Plan for all three.

What did Cloudflare announce during Birthday Week 2026?

Three launches matter for anyone sending automated traffic to websites, and Cloudflare's Birthday Week wrap-up of October 5 lists all three. They build on tools Cloudflare already had before the week: AI Crawl Control, Web Bot Auth, Markdown for Agents, and the Search, Agent, and Training controls it introduced in July.

Source: Cloudflare, "The Internet has a second audience" (September 30, 2026), the Birthday Week wrap-up (October 5, 2026), and the linked launch posts.

Cloudflare frames the strategy as three jobs for site owners: "see who's visiting, decide who gets in, and charge for access." Birthday Week added two new ways to do the third. Agent builders should read the list in reverse. Each step is a new question your traffic will be asked.

Why is Cloudflare doing this now?

Because the mix of who reads the web has changed. In its September 30 post, Cloudflare says "more than half of Internet traffic wasn't human" and that human traffic in categories such as retail, computer software, and financial services fell "as much as 40% in less than one year" (Cloudflare, 2026). It also reports that AI training grew from 22% of crawler requests in spring 2025 to 52% in June 2026, based on the purpose each crawler states.

Those numbers come from one network, Cloudflare's, so treat them as a large sample rather than the whole web. The direction matches what we described when Cloudflare changed its defaults on September 15: sites stopped treating every bot the same way. Our breakdown of the September 15 crawler defaults covers that first step. Birthday Week adds new payment options on top of it.

How do x402 payments work on Cloudflare?

The Monetization Gateway uses the HTTP 402 Payment Required status code. When an agent requests a priced resource without paying, the site answers 402 with payment instructions. The agent pays and retries, and the site returns the resource. One of Cloudflare's launch examples, the PDF-generation service API2PDF, quotes an agent the maximum price a request could cost. Once the agent pays, API2PDF "fulfills the request and settles only the actual consumption" (Cloudflare, 2026).

The status code itself is old. HTTP's core specification still lists 402 as "reserved for future use" (RFC 9110, 2022), and x402 is one attempt to give it a job. Payments run on the x402 protocol and settle in USDC on the Base blockchain through Coinbase's x402 facilitator. Sellers can set prices on any part of a request, including the URL, headers, or query parameters. Access is limited for now: the gateway is "available in closed beta to eligible U.S.-based sellers and buyers."

We have seen the same flow work from the seller side. Massive's Search endpoint is listed on Orthogonal's x402 gateway, and when we called it in October 2026, one unpaid request returned 402 with the price in a header. Paying and retrying returned the results. The full walkthrough is in our Orthogonal spotlight.

What is the difference between Pay Per Use and pay per crawl?

Pay per crawl, which Cloudflare introduced in 2025, charges for access. We covered how it started in the closing web and pay per crawl. Pay Per Use charges for use. In Cloudflare's words, "Paying for use ties the price to the value the buyer actually gets" (Cloudflare, 2026).

The mechanics are worth reading closely. Usage is self-reported: buyers send the time of each use, the URL the content came from, and an event ID. Cloudflare checks that each reported use maps to an enrolled publisher, charges the buyer, and pays publishers monthly. During the beta, Cloudflare is working directly with each buyer and each publisher who opts in, and it has not named the buyers.

For an AI company, that means the cost of content can now depend on how often a page is used in answers, not only on how often it is fetched. For a data team, it is a new line item to plan around if your sources enroll.

Cloudflare's Web Search API returns links, not pages

On October 2, Cloudflare added a Web Search API to AI Gateway. It lets an agent search the web from inside a model call, with three provider partners at launch: Ceramic.ai, Exa, and Linkup. Cloudflare passes through the partners' list prices "without any additional markup," and says the partners are committed to identifying their crawlers, respecting robots.txt, and providing the source of search results.

It returns search snippets and links, not rendered pages. An agent that needs the full page behind a link, or the results a searcher sees in a specific country, still needs a way to fetch that page.

How does Web Bot Auth work?

With Web Bot Auth, a bot signs its requests instead of relying on a user-agent string anyone can copy. Cloudflare's Web Bot Auth docs describe four steps. The operator generates an Ed25519 key pair, publishes the public key in a key directory at /.well-known/http-message-signatures-directory, and registers that directory with Cloudflare as a verified bot. After verification, the bot signs each request using HTTP Message Signatures (RFC 9421), covering at least the @authority component, which is the domain being requested. Web Bot Auth predates Birthday Week, and Cloudflare says operators including OpenAI, Google, and AWS already sign their agents' requests this way.

The method builds on two IETF drafts, one for the key directory and one for the signing protocol. Neither is a finished standard yet. For an agent operator, the main cost is key management: a private key to protect, a directory to host over HTTPS, and a registration per network that verifies bots this way.

What changes for teams that run AI agents or data pipelines?

Requests now fall into one of three lanes, and each one needs a plan.

1. Identified traffic. Sites behind Cloudflare can already verify which bot is calling through Web Bot Auth, as described above, and treat search, agent, and training traffic differently. If you run a crawler that identifies itself, expect each site to apply a different policy to it. Log which lane each source puts you in.
2. Priced traffic. Some sources will answer with 402 instead of content. Decide in advance which sources are worth paying for, set a budget per source, and treat a 402 as a business decision, not an error to retry.
3. Open public pages. Many sites still serve public pages to ordinary visitors without identity checks or prices. For that lane, the questions are the familiar ones: does the page you fetch match what a local user sees, and does your traffic respect each site's limits?

Transparency cuts both ways. Massive publishes what its network blocks and how customers can block destinations themselves. Our domain blocking transparency report explains the controls. If your agents need live public pages from a specific location, Massive's device-access network returns clean HTML or markdown from any public source, in any location.

What to watch next

• Who joins the paid lanes. Pay Per Use buyers are unnamed during the beta. The first named buyers will show what AI companies are willing to pay for.
• Monetization Gateway outside the U.S. Cloudflare says support for new geographies is "on the way" (Cloudflare, 2026). Until then, 402 pricing through Cloudflare is a U.S. arrangement.
• Standards. The rails "run on open standards like x402 and Web Bot Auth, so anyone can build on them," per Cloudflare.

The bottom line

• Cloudflare's Birthday Week 2026 launches widen the priced lane with x402 payments and Pay Per Use, so bot traffic now splits three ways: identified, priced, and open. Its own data says agent traffic grew more than 1,700% in a year.
• Web Bot Auth and the Search, Agent, and Training controls were already available before Birthday Week. Of the new parts, x402 payments through the Monetization Gateway and Pay Per Use are in beta, and the Web Search API is available in AI Gateway.
• Build for all three lanes: identify where sites ask you to, budget for 402 responses, and make sure your fetches of open public pages match what local users see.

Sources

• Cloudflare, "The Internet has a second audience", September 30, 2026
• Cloudflare, "Monetization Gateway beta: charge AI agents for consumption with HTTP 402", 2026
• Cloudflare, "Pay Per Use: when AI uses your work, you should get paid", 2026
• Cloudflare, "Introducing Web Search API via AI Gateway", October 2, 2026
• Cloudflare, "Everything we launched during Birthday Week 2026", October 5, 2026
• x402, x402.org
• Cloudflare Docs, "Web Bot Auth"
• IETF, RFC 9421: HTTP Message Signatures, and RFC 9110: HTTP Semantics, 402 Payment Required
• IETF, draft-meunier-web-bot-auth-architecture