Identify every AI crawler at the edge — then allow, block, or charge for access. Your content is leverage; Cloudflare makes it a policy.
AI crawlers ingest your site to train models and answer questions — often without a click back. Cloudflare sees the bots the rest of the internet can’t, by behaviour, not just user-agent, so you set the terms.
Robots.txt is a request. The edge is enforcement.
With Cloudflare’s pay-per-crawl, an AI company that wants your content gets a 402 and a price, not a free ride. Allow the partners you choose, block the rest, and meter everyone in between.
From free-for-all to a market you control.
Toggle each AI crawler’s policy and your robots.txt + edge rules update instantly. The verdict feed shows simulated live requests.