Scrapeless Wiki

What Is Dynamic Content? – Definition and How It Works

Informational P2 what is dynamic content

Learn what makes content dynamic, the difference between server-generated and client-assembled, and how each affects collection.

Dynamic content is generated per request rather than served from a fixed file. The term covers two quite different situations, and conflating them causes a lot of wasted effort.

1. Two Kinds of Dynamic

Server-generated β€” the server builds the HTML per request, from a database, personalisation, or location. The response is complete; it simply was not a static file.

Client-assembled β€” the server sends a shell and the browser fetches data and builds the page. The response is incomplete without script execution.

Server-generated Client-assembled
Content in the response Yes No
A plain fetch suffices Yes No
Varies per request Yes Yes

For collection, only the second requires a different approach β€” and it is routinely assumed whenever a page "looks dynamic".

2. What Makes Content Vary

  • Personalisation by account or history.
  • Location, affecting currency, catalogue, and language.
  • Device, changing layout and sometimes content.
  • Time β€” availability, pricing, and recency.
  • Experiments, serving different variants to different visitors.
  • Live data such as stock levels and prices.

Every one of these applies to both kinds, which is why "dynamic" says nothing on its own about how to collect it.

3. The One Diagnostic

curl -s "https://example.com/page" | grep -c "a value visible in the browser"

Non-zero: the content is server-generated and a plain fetch works. Zero, with the value visible in a browser: it is client-assembled.

That single command decides the entire approach and takes seconds.

4. Implications for Comparability

Because dynamic content varies, an observation is only meaningful with its context: location, device, time, and any session state.

A price collected from one region is not the same observation as one from another. Recording that context is what makes a series comparable rather than a mixture.

5. Experiments Are the Hidden Variable

A site running an experiment serves different variants to different visitors, sometimes with different content rather than only different styling.

A collector can land in either arm, and nothing announces which. Where variance between runs is unexplained, an experiment is a plausible cause worth considering before assuming a fault.

6. Approaches by Kind

Situation Approach
Server-generated Fetch and parse
Client-assembled, endpoint reachable Call the endpoint
Client-assembled, state embedded in HTML Fetch and parse the embedded JSON
Client-assembled, neither available Render in a browser

7. Validate Against Silent Emptiness

Both kinds can return a complete-looking page with no content β€” an empty state, a challenge, or a shell. Assert on record counts and required fields so a run that collected nothing fails rather than writing a quiet zero.

8. Summary

Dynamic content is generated per request, either on the server β€” where a plain fetch works β€” or in the browser, where it does not.

One grep distinguishes them. Beyond that, dynamic means varying, so location, device, time, and session state are part of every observation and must be recorded with it.