What Is XPath? – Definition and How It Works
Learn what XPath is, how axes and predicates select nodes, how it compares with CSS selectors, and which expressions survive page changes.
XPath is a language for addressing parts of an XML or HTML document. It describes a path through the tree, with conditions, and returns the nodes that match.
1. What Is XPath?
XPath treats a document as a tree and selects nodes by their position, name, attributes, or content.
- Key idea: navigation in any direction, including upward, which CSS selectors cannot do.
- Mechanism: an expression of steps separated by slashes, each optionally filtered by a predicate.
- Goal: precise selection from structured markup.
Example
//div[@class="product"]/h2/text() the heading text of every product block
//a[contains(@href, "/item/")]/@href every item link's URL
//td[text()="Price"]/following-sibling::td first cell after the Price label
2. The Parts of an Expression
| Part | Meaning |
|---|---|
/ |
A step down one level |
// |
A step down any number of levels |
@attr |
An attribute rather than an element |
[...] |
A predicate — a condition or an index |
axis:: |
A direction, such as parent, following-sibling, ancestor |
text() |
The text node inside an element |
Indexes start at one, not zero, which is a frequent source of off-by-one confusion.
3. Useful Predicates
[@id="main"]— exact attribute match.[contains(@class, "price")]— substring match, useful when classes are combined.[normalize-space(text())="Total"]— text match ignoring surrounding whitespace.[position()=1]or[1]— the first match.[not(@hidden)]— negation.
4. XPath Versus CSS Selectors
| Capability | CSS | XPath |
|---|---|---|
| Select by class, id, attribute | Yes | Yes |
| Select by text content | No | Yes |
| Traverse upward to an ancestor | No | Yes |
| Sibling navigation | Limited to following siblings | Both directions |
| Readability | Usually shorter | More verbose |
| Support | Universal in browsers | Universal in parsers; browser support via specific APIs |
The two overlap heavily. XPath earns its place when the anchor is text, or when the target sits above or before a known element.
5. Writing Expressions That Survive
- Anchor on stable attributes —
id,data-*,name— before class names. - Avoid absolute paths from the root; a single inserted wrapper breaks them.
- Prefer a text anchor plus a relative step over a long positional chain.
- Treat hashed or generated class names as unstable; they change on every deploy.
- Test against several pages of the same type, not just one.
6. Common Misconceptions
- "XPath is only for XML." HTML parsers expose it routinely, and it is widely used for web extraction.
- "
//is free." It searches the whole subtree and can be markedly slower on large documents. - "Indexes are zero-based." They start at one.
- "XPath sees rendered content." It queries the parsed document. Content added later by scripts must have been present when the document was captured.
7. Real-World Examples
- Selecting a value by its label — find the cell containing "Price", then take the next cell.
- Extracting every link matching a URL pattern from a listing page.
- Climbing from a matched element to its containing row to collect the whole record.
- Pulling attribute values, such as image sources or data attributes, in one expression.
8. Summary
XPath addresses nodes in a document tree using steps, axes, and predicates, and it can do two things CSS selectors cannot: match on text and traverse upward.
Its durability depends on what it anchors to. Stable attributes and text anchors survive redesigns; absolute paths and generated class names do not.