Scrapeless Wiki

What Is a CDN? – Definition and How It Works

Informational P2 what is a cdn

Learn what a content delivery network does, how edge caching and origin shielding work, and what it means for anyone fetching pages.

A content delivery network is a distributed set of servers that sit between visitors and an origin server, caching content close to users and absorbing traffic the origin would otherwise handle.

1. How Requests Reach the Edge

DNS resolves a site's hostname to an address belonging to the network rather than to the origin. The visitor connects to the nearest edge location, and that server either answers from cache or fetches from the origin.

That indirection is why the address behind a site is usually the network's, not the origin's.

2. Caching

Concept Meaning
Cache hit Served from the edge; the origin never sees it
Cache miss Fetched from the origin, then stored
TTL How long an object may be reused
Purge Explicit invalidation before expiry
Origin shield An intermediate tier that absorbs misses from many edges

Cache behaviour is driven by response headers, which is why a misconfigured Cache-Control produces either stale content or no caching at all.

3. What Else the Edge Does

Modern networks do far more than cache:

  • TLS termination, so the handshake completes near the user.
  • Compression and image optimisation.
  • Bot management and rate limiting.
  • Web application firewall rules.
  • Routing across regions and failover.
  • Edge compute, running code before requests reach the origin.

The consequence is that many responses are generated at the edge and never involve the origin at all.

4. What It Means When Fetching Pages

  • The address you connect to is the network's, so it says nothing about where the site is hosted.
  • Cached responses may be stale, which matters when collecting time-sensitive values.
  • Geography changes what you get, since routing and content can both vary by location.
  • Refusals often come from the edge, which is why error pages carry the network's branding rather than the site's.
  • Headers reveal the layer β€” cache status headers indicate whether a response came from cache.

5. Why Content Varies by Location

Edges serve region-appropriate content, and sites frequently vary pricing, catalogue, and language by region. Two collectors in different countries can therefore observe genuinely different pages.

Any pipeline touching regional content should pin and record the egress location, or it is silently mixing observations.

6. Freshness and Collection

A cached response can be older than it appears. Where currency matters, check whether the response indicates a cache hit and how old it is, and treat a long-cached value as an observation about the cache rather than about the source.

7. For Site Operators

Set cache headers deliberately per content type, use purging for updates that must appear immediately, and remember that edge rules β€” rate limits, firewall rules, bot management β€” apply before your application sees anything.

8. Summary

A content delivery network caches and serves content from edge locations near users, terminating TLS and applying security and routing rules before requests reach the origin.

For anyone fetching pages, the practical consequences are that the address is the network's, responses may be cached, content varies by region, and refusals frequently come from the edge rather than the site.