Scrapeless Wiki

What Is AWS WAF? – Definition and How It Works

Informational P2 what is aws waf

Learn what AWS WAF is, how Bot Control and its challenge flow work, and the signals that reveal it on a site.

AWS WAF is Amazon's web application firewall. It filters HTTP requests by rule before they reach an application, and its Bot Control add-on adds bot detection with browser challenges.

1. What It Actually Is

A rule engine, not a single product. An operator composes rules that inspect requests and take an action:

Rule inspects Examples
Request attributes IP, country, header, URI, query, body, method
Rate Requests per interval per key
Managed rule groups AWS-curated sets, including Bot Control
Labels Applied by earlier rules, matched by later ones

Actions are Allow, Block, Count, CAPTCHA, or Challenge. The last two are what make it relevant to automated access.

2. Bot Control Is the Part That Detects Bots

The base WAF is rules on request attributes. Bot Control is the managed rule group that classifies traffic, in two tiers:

  • Common β€” catches self-identifying and obvious automation: known bot user agents, missing browser headers, simple signatures.
  • Targeted β€” adds browser fingerprinting, client-side JavaScript interrogation, and behavioural signals aimed at automation that is trying not to be seen.

The tier decides your experience entirely. Common-tier protection often passes with an honest, consistent, well-behaved client. Targeted tier requires a real browser that can execute the challenge.

3. The Challenge Flow

Challenge returns a page containing a JavaScript challenge rather than the content. The script runs, collects signals, posts them, and on success a token cookie is set; subsequent requests carrying that token are served normally.

CAPTCHA is the same shape with a human-interaction puzzle added.

Consequence for a plain HTTP client: it receives the challenge page, cannot execute it, and β€” if it parses the response as content β€” extracts nothing while seeing a 2xx status. That is the silent-failure shape to guard against.

4. Signals That Reveal It

Verified detection signals for AWS WAF, ordered by strength:

Class Signal Confidence
Cookie aws-waf-token Definitive
Cookie awswaf* Definitive
URL /challenge.js Definitive
URL /inputs?client=browser High
URL /telemetry High
URL /verify High
URL / content awswaf string Moderate

The aws-waf-token cookie is the giveaway β€” it is namespaced, stable, and present once a challenge has been completed.

5. Distinguishing It From CloudFront Alone

AWS WAF attaches to CloudFront, an Application Load Balancer, or API Gateway, so a site on AWS may show CloudFront headers with no WAF at all.

The distinction: CloudFront presence is visible in response headers and tells you about the CDN. WAF presence is indicated by the token cookie and challenge endpoints above. Seeing one does not imply the other.

6. What This Means in Practice

  • Rate rules are common and cheap for operators to enable, so per-address request limits should be assumed.
  • Geographic rules are common too, so a block that follows egress location rather than request rate points here.
  • Count mode exists, and many deployments run Bot Control in Count first β€” meaning the vendor is present and not blocking.
  • A challenge means a browser is required, and no header tuning substitutes for executing the script.
  • 429 and 403 both appear, depending on which rule matched, so the status alone does not identify the cause.

7. Reading the Response Correctly

Because Challenge returns a 2xx with a challenge body, treat structural validation as mandatory: assert that the expected content markers exist rather than that the request succeeded. A pipeline that checks only the status code will record challenge pages as data.

The same applies to Count mode in reverse β€” traffic passes, and nothing indicates that it was scored.

8. Summary

AWS WAF is a rule engine in front of an AWS-hosted application, with Bot Control adding bot classification in a Common tier that catches obvious automation and a Targeted tier that fingerprints browsers.

Its Challenge and CAPTCHA actions return challenge pages with success status codes, so validate content rather than status; and identify it from the aws-waf-token cookie and the /challenge.js, /telemetry, and /inputs?client=browser endpoints rather than from CloudFront headers, which indicate only the CDN.