Cloudflare updates default settings to restrict AI crawlers on ad-supported websites

Web infrastructure firm Cloudflare plans to automatically block AI model training bots and automated agents on ad-supported pages starting September 15, 2026. Chief Strategy Officer Stephanie Cohen noted that non-human traffic now accounts for nearly sixty percent of web activity, while scrapers extract data with declining reader referral rates. To address this imbalance, Cloudflare is developing machine-to-machine payment rails using HTTP 402 status codes to allow publishers to bill AI platforms on a per-use basis.