How does Cloudflare treat Web Bot Auth signed agents?

Since July 1, 2026, agents that sign requests with Web Bot Auth count as verified bots on Cloudflare, and every verified bot is labelled either Direct — operated by a single narrow operator on its own infrastructure — or Intermediary — an agentic service that many different end users drive, such as a personal AI agent shopping for a person. Bot Management customers can match cf.bot_management.signed_agent (true when a request carries a valid signature) and cf.verified_bot_category to write allow or block rules. A bot that abuses the trust loses its verified status. Cloudflare is also experimenting with forwarding end-user information in the RFC 7239 Forwarded header so sites can apply policy to the responsible person.

Which AI agents use Web Bot Auth today?

Adoption is early. An August 2026 probe of six major agent domains found keys served at /.well-known/http-message-signatures-directory by chatgpt.com and browserbase.com, with the other four returning nothing. OpenAI's ChatGPT agent traffic is therefore verifiable; Meta's Muse, launched September 8, 2026, is not — Amazon's stated reason for blocking it on September 20 was that the agent 'doesn't identify itself' when browsing. Expect the share of signed agent traffic to grow as Cloudflare's default policies for new domains (since September 15, 2026) block unverified Agent traffic on pages with ads.

Is Web Bot Auth the same as robots.txt or a user-agent allowlist?

No. robots.txt expresses your preferences to compliant crawlers but proves nothing about who is asking, and several agent operators state it may not apply to user-initiated fetches. A user-agent allowlist can be walked through by any client that types 'Googlebot' or 'ChatGPT-User' in a header. Web Bot Auth answers a different question — who signed this request — and lets you allow a declared agent while blocking imitators. You still need robots.txt for preference signalling and Content Signals for legal reservation of rights; Web Bot Auth is the enforcement-grade identity layer on top.

Quick Answer

What Is Web Bot Auth? Signed AI Agents Explained (2026)

Q: What is Web Bot Auth?

Web Bot Auth is a mechanism for a crawler or AI agent to prove its identity cryptographically instead of by user-agent string. The operator generates an Ed25519 key pair, publishes the public key as a JSON Web Key Set at /.well-known/http-message-signatures-directory over HTTPS, registers that directory with Cloudflare, and signs every request using HTTP Message Signatures, attaching Signature, Signature-Input and Signature-Agent headers. The receiving edge validates the signature against the registered key. Because a user-agent header is a claim anyone can type, and reverse DNS and IP allowlists rot as operators move infrastructure, a signature is the only identity signal that stays valid over time.

Published: September 22, 2026

The short answer

Web Bot Auth is how an AI agent proves who it is: it signs each HTTP request with a private key whose public half is published at a well-known URL, and the edge verifies the signature. Cloudflare has treated signed agents as verified bots since July 1, 2026, labelling them Direct or Intermediary. It matters now because the September 2026 Amazon–Meta fight over the Muse agent turned on one word — Muse “doesn’t identify itself” — and Web Bot Auth is the standard answer to that complaint.

Why user-agent strings stopped being enough

Every earlier bot-control method relies on a claim the client makes about itself:

Signal	Problem
User-agent string	Any HTTP client can send `GPTBot` or `Googlebot`; scrapers spoof Googlebot precisely because sites allowlist it
Reverse DNS / IP ranges	Degrade as operators move infrastructure; Cloudflare documented in August 2025 an undeclared Perplexity crawler rotating IPs and ASNs after its declared agent was blocked
robots.txt	A preference, not an identity; OpenAI and Perplexity state it may not apply to user-initiated agent fetches
Browser fingerprinting	Agents that drive a real Chromium (Meta Muse, Perplexity Comet, ChatGPT’s visual browser) look like people

The last row is the hard case. A self-identifying bot is easy to gate; an agent piloting a genuine browser on a human’s behalf is not. Web Bot Auth gives that agent a way to declare itself that cannot be forged.

How Web Bot Auth works

Key pair. The agent operator generates an Ed25519 key pair.
Publish. The public key goes into a JSON Web Key Set served over HTTPS at /.well-known/http-message-signatures-directory on the operator’s domain.
Register. The operator registers that directory with Cloudflare (and any other edge that supports the scheme).
Sign. Each request carries HTTP Message Signatures headers: Signature (the signature), Signature-Input (what was signed and with which key) and Signature-Agent (the directory URL identifying the operator).
Verify. The edge fetches the directory, validates the signature, and marks the request as a signed agent. Invalid or missing signatures fall back to ordinary bot scoring.

Because the key, not the network, carries identity, an operator can change hosting providers, IPs and even user-agent strings without breaking verification.

Cloudflare’s model since July 1, 2026

Cloudflare reorganised bot handling on July 1, 2026 from a single “AI bot” label into behaviours — Search, Agent, Training, Transact, Data Collection, Security Testing, SEO, Ads Verification, Social and Link Preview, Feed Fetching, Monitoring and Operations — and made three of them (Search, Agent, Training) presets on every plan. Web Bot Auth slots in as the identity layer:

Concept	Meaning
Verified bot	Identity confirmed — by Cloudflare’s existing verification programme or by a valid Web Bot Auth signature
Direct	A verified bot run by one narrow operator on its own infrastructure; only that operator can present as it (e.g. a search crawler)
Intermediary	A verified agentic service many end users drive (e.g. a personal agent shopping for thousands of people) — introduces transitive trust: you may trust the operator without trusting every user
`cf.bot_management.signed_agent`	Boolean, true when the request carries a valid Web Bot Auth signature (Bot Management plans)
`cf.verified_bot_category`	String for segmenting verified traffic by purpose (Bot Management plans)
`Forwarded` (RFC 7239)	Experimental: end-user information forwarded by Intermediary agents so sites can apply policy to the responsible person

Abuse has a consequence: a bot that misbehaves “will lose the Verified status” and with it the access verified bots receive.

Defaults changed too. From September 15, 2026, newly onboarded Cloudflare domains block Training and Agent traffic on pages that display ads while allowing Search — so unsigned, unverified agents are now blocked by default on a growing share of the ad-supported web, while signed agents can be allowed selectively.

Who has adopted it

Slowly, so far. A probe of six agent domains in August 2026 found keys at the well-known path on chatgpt.com and browserbase.com; the other four served nothing. Vercel’s WAF documentation now supports matching the Signature-Agent header, and Akamai and Vercel have their own agent-identity schemes (compared in Akamai KYA vs Vercel Passport vs Cloudflare).

The gap is visible in the Meta Muse case. Muse, launched September 8, 2026, browses through a Chromium instance in a per-user VM and, per Amazon, does not identify itself as an agent. Amazon blocked it on September 20 citing exactly that, alongside credential-storage concerns. Had Muse signed its requests, Amazon’s “undisclosed third party” argument would be moot — though its commercial objection (agents bypass its ad business) would remain. See Why Amazon blocked Meta Muse.

What it means for site owners

Allow declared, block undeclared. With Bot Management, cf.bot_management.signed_agent lets you admit ChatGPT’s agent while challenging an unsigned headless browser doing the same thing. On Free and Pro, use the Agent preset plus explicit user-agent tokens.
Treat Intermediary differently from Direct. An intermediary agent is a crowd; rate-limit per forwarded end user where available, not per operator.
Keep the other layers. robots.txt with Content Signals still expresses preferences and reserves rights; terms of service still matter (Amazon’s Muse block rests on its Conditions of Use after the Ninth Circuit’s August 4, 2026 Perplexity ruling closed the hacking-law route). Full playbook: How to control AI shopping agents on your website.

What it means for agent builders

If you ship an agent that fetches pages or buys things for users, sign your requests. It costs a key pair and a static JSON file, and it is quickly becoming the difference between being allowed in by default and being blocked by default. Publish an operator page explaining what your agent does, honour robots.txt for non-user-initiated crawling, and expect sites to ask for the RFC 7239 end-user context as the Intermediary model matures.