How Does JavaScript Rendering Work?

Follow a client-rendered page from empty shell to finished DOM: bundle execution, data fetching, the critical rendering path, and hydration.

Understanding when content appears is the difference between a scraper that reads a finished page and one that captures an empty container. The sequence is well defined, and the useful thing is knowing which step you are waiting on.

1. The Sequence

For a client-rendered page:

1. HTML arrives — a shell with a mount point and script tags.
2. The parser encounters the bundle and requests it.
3. The bundle executes, initialising the framework.
4. The application requests data from an API.
5. The response arrives, often hundreds of milliseconds later.
6. The framework renders, creating DOM nodes.
7. The browser lays out and paints the new content.

Content exists only after step 6. Every step before it is a period during which the page is visibly and structurally empty.

2. What the Server Actually Sent

<!DOCTYPE html>
<html>
  <body>
    <div id="root"></div>
    <script src="/static/js/main.8f2a.js" defer></script>
  </body>
</html>

That is the complete response. An HTTP client receives this, parses it successfully, and finds nothing — with a 200 status and no error.

3. The Critical Rendering Path

Between receiving HTML and painting pixels, the browser:

• Parses HTML into the DOM
• Parses CSS into the CSSOM
• Combines them into a render tree of visible nodes
• Computes layout — the geometry of every box
• Paints into layers
• Composites layers into a frame

Scripts interrupt this. A classic <script> blocks parsing while it downloads and executes. defer postpones execution until parsing completes; async runs it as soon as it arrives, whenever that is.

For a client-rendered app the whole path runs twice in effect: once on the empty shell, and again after the framework inserts real content.

4. Where the Data Comes From

Step 4 is the one that matters most for extraction. The application makes an HTTP request — usually to a JSON endpoint on the same origin:

GET /api/v2/products?page=1

That request is visible in DevTools → Network → Fetch/XHR. It is the actual source of the page's content, and it is almost always a better extraction target than the DOM built from it: smaller, already structured, and not coupled to any visual redesign.

Some frameworks skip the round trip by embedding the data in the initial HTML:

<script id="__NEXT_DATA__" type="application/json">{"props":{...}}</script>

When that is present, the data is in the response you already have.

5. Hydration

Server-rendered frameworks invert the order. The server runs the same components and sends finished HTML, so content is present immediately. The bundle then loads and hydrates — attaching event handlers to markup that already exists.

The consequence for extraction is favourable: content is in the initial response, and the embedded state blob usually is too. A "React site" is not automatically a browser job — check whether it is hydrated before assuming it is client-rendered.

6. Waiting Correctly

Inside a browser, the timing problem persists. Reading the DOM at load fires before the data request has returned.

page.wait_for_selector(".product", timeout=15000)   # wait for the thing you need

That is the reliable form: wait for the specific element whose presence proves rendering completed.

The alternatives are worse. Fixed sleeps guess. networkidle waits for network quiet, which never arrives on pages with polling, analytics beacons or open WebSockets — a hang rather than a failure, and one that consumes the full timeout every time.

For content that streams in progressively, waiting for the container is not enough; wait for a stable count of items or for a completion marker, or you will capture a partial render and store it as complete.

7. Telling Which Kind of Page You Have

Three checks, in ascending effort:

• curl the URL and search for a string you can see on the page. Ten seconds, definitive.
• View source, not Inspect. Source is what the server sent; Inspect shows the live DOM.
• Disable JavaScript and reload. What remains is what an HTTP client gets.

8. Summary

A client-rendered page arrives as a shell, loads a bundle, executes it, requests data, renders the response into DOM nodes, and only then paints. Content exists after the render step, which is why a plain fetch sees nothing and why timing matters even inside a browser.

Hydrated server-rendered pages behave very differently — the content, and usually the data, are in the initial response.

The highest-value move is to find the data request in step 4. That endpoint returns the same information already structured, without the bundle, the render or the browser. Where a browser is genuinely required, wait on the specific element you need rather than on a clock or on network idle.

Potential Related Glossary

• Headless Browser
• Parsing
• Unstructured Data
• Web Crawling