Scrapeless Wiki

What Is the DOM? – Definition and How It Works

Informational P1 what is the dom

Learn what the Document Object Model is, how markup becomes a live tree, how scripts mutate it, and why the DOM often differs from the served HTML.

The Document Object Model is the in-memory tree a browser builds from a page's markup, and the interface scripts use to read and change it. It is the live state of the document, which is not the same thing as the HTML the server sent.

1. What Is the DOM?

It is an object representation of a document.

  • Key idea: markup becomes a tree of nodes that code can query and modify.
  • Mechanism: the parser builds the tree; scripts mutate it; the renderer paints from it.
  • Goal: make a document programmable rather than static.

The Tree

document
└── html
    β”œβ”€β”€ head β†’ title, meta, link
    └── body
        β”œβ”€β”€ header β†’ nav β†’ a, a, a
        └── main β†’ section β†’ h1, p, ul β†’ li, li

2. How a Page Becomes a Tree

  1. Bytes arrive and are decoded into characters using the declared encoding.
  2. The parser tokenises the markup and constructs nodes.
  3. Malformed markup is repaired according to the specification, which is why a browser's tree can differ from the literal source.
  4. Scripts run and may insert, remove, or alter nodes.
  5. Styles are resolved and the result is painted.

Step three explains a common surprise: unclosed tags and misnested elements produce a tree that no longer matches the text you read in the source.

3. Served HTML Versus Live DOM

Source What it contains
Response body from a plain fetch Only what the server sent
DOM after load The parsed tree, plus everything scripts added

On a client-rendered page the served body may contain almost nothing but a container and a script bundle. Anything that inspects only the raw response sees an empty shell, while the same page in a browser is full of content.

This single distinction determines whether a plain HTTP request is enough or a browser is required.

4. Working With the Tree

Queries use CSS selectors or XPath, and traversal uses parent, child, and sibling relationships. Reading text, attributes, and computed styles are all ordinary operations once the tree exists.

Mutation is equally routine, which is why a page's final state can differ from its initial one after user interaction β€” expanding a section, loading more results, or opening a dialog all change the tree.

5. The Shadow DOM

Some components encapsulate their internals in a shadow tree attached to a host element. Ordinary queries do not cross that boundary, so an element visible on screen can be unreachable from a document-level selector.

Reaching inside requires querying the host's shadow root explicitly, and closed shadow roots do not expose one at all.

6. Common Misconceptions

  • "The DOM is the HTML." It is the parsed, repaired, script-modified state. They coincide only on fully static pages.
  • "View source shows the DOM." It shows the served bytes. Developer tools show the tree.
  • "If it renders, a fetch will find it." Client-rendered content exists only after scripts run.
  • "Selectors reach everything." Shadow boundaries and same-origin restrictions on frames both limit reach.

7. Real-World Examples

  • A product page whose price arrives from a later API call and is absent from the served HTML.
  • A table that only populates after a filter is applied.
  • A component whose markup lives in a shadow root and returns nothing to a document-level query.
  • A page whose source is invalid markup, where the browser's repaired tree differs structurally from the text.

8. Summary

The DOM is the live, in-memory tree a browser builds from markup and scripts modify continuously. It is the thing selectors query and the thing rendering reflects.

The practical consequence is the gap between served HTML and live DOM: whenever content appears only after scripts run, extraction needs the tree rather than the response body.