Scrapeless Wiki

How Does Cloudflare Bot Detection Work? – Signals and Decisions

Informational P1 how does cloudflare bot detection work

Learn how edge bot detection scores a request: network reputation, TLS and HTTP fingerprints, browser challenges, and the actions a site can take.

Edge bot detection happens before a request reaches the origin server. The decision is made from what the connection itself reveals, plus what an in-browser challenge can confirm, and results in an action the site operator configured.

1. Where the Decision Happens

A request passing through a reverse proxy is evaluated at the edge. The origin never sees blocked traffic, which is the point: filtering costs the site nothing in origin capacity.

The evaluation produces a score, and the site's rules map score ranges to actions β€” allow, challenge, log, or block.

2. Connection-Level Signals

These are available before any JavaScript runs, which is why a plain scripted request can be identified without ever loading a page.

Layer What is examined
Network Address reputation, autonomous system, whether the range is datacenter or consumer
TLS The handshake's cipher list, extensions, and their order β€” a fingerprint of the client library
HTTP/2 Frame settings, priority, and header ordering, which differ between real browsers and libraries
Headers Presence, order, and plausibility of headers a real browser always sends

A mismatch between layers is the strongest signal of all: a user agent claiming to be a mainstream browser, sent over a TLS handshake that no such browser produces, is self-contradictory.

3. Browser-Level Challenges

When connection signals are inconclusive, an interstitial page runs code in the client and reports back.

  • Environment consistency β€” do the browser's own reported properties agree.
  • Rendering behaviour β€” do drawing and font operations produce expected results.
  • Timing β€” does the client complete work at a plausible speed.
  • Interaction β€” does subsequent behaviour look like a person using the page.

Passing yields a clearance cookie, which lets subsequent requests skip the interstitial for a period.

4. The Actions a Site Can Choose

  • Allow β€” pass through, optionally with a score header for the origin to use.
  • Managed challenge β€” issue an interstitial, sized to the suspicion level.
  • Log β€” record the verdict without changing behaviour, used while tuning.
  • Block β€” refuse with an error page carrying a diagnostic code.

Because these are configured per site, identical traffic can pass one domain and be blocked by another.

5. Verified Bots Are Treated Separately

Search and AI crawlers that publish their address ranges and honour robots.txt can be identified and allowed explicitly. That is why a search engine's fetcher is not caught by rules that stop unidentified automation, and why declaring a stable, documented user agent matters for anyone operating a legitimate crawler.

6. Why Identical Requests Get Different Outcomes

  • Reputation is dynamic. An address's history changes hour to hour.
  • Rules are per-site. Two domains behind the same edge can be configured very differently.
  • Scores are probabilistic. Borderline traffic falls on either side of a threshold between runs.
  • Clearance is stateful. A session that already passed carries a cookie that later requests benefit from.

7. What It Means for Legitimate Automation

Consistency is what survives evaluation. A client whose TLS fingerprint, HTTP behaviour, headers, and JavaScript environment all belong to the same real browser presents no contradiction to score against. Contradiction β€” not automation itself β€” is what the system is built to find.

8. Summary

Edge bot detection scores each request from network reputation and protocol fingerprints, escalates uncertain cases to an in-browser challenge, and applies whichever action the site configured for that score.

There is no single test to pass. The decision is a weighted judgement over many weak signals, which is why outcomes vary between sites, between addresses, and between runs.