Scrapeless Wiki

What Is Cursor-Based Pagination? – Definition and How It Works

Informational P2 what is cursor-based pagination

Learn how cursor pagination works, why it is stable under changing data, how cursors are constructed, and what it gives up compared with offsets.

Cursor pagination marks a position in a result set with an opaque token rather than counting rows from the start. The next page resumes from that marker.

1. How It Works

GET /items?limit=100
→ { "items": [...], "next_cursor": "eyJpZCI6NDE5Mn0" }

GET /items?limit=100&cursor=eyJpZCI6NDE5Mn0
→ { "items": [...], "next_cursor": null }

A null or absent cursor signals the end. The client never computes a position; it only carries forward what the server returned.

2. Why It Is Stable

Offset pagination counts rows. If a record is inserted or deleted between two requests, every subsequent offset shifts, and the client silently skips or repeats rows.

A cursor encodes a position in a sort order rather than a count, so insertions and deletions elsewhere in the set do not move it.

3. How Cursors Are Built

A cursor typically encodes the sort key values of the last returned row, and the query resumes after them:

SELECT * FROM items
WHERE (created_at, id) > (:last_created_at, :last_id)
ORDER BY created_at, id
LIMIT 100;

The identifier is included as a tiebreaker, because timestamps collide. A sort key without a unique tiebreaker produces skipped or repeated rows at page boundaries.

4. Why It Performs Better at Depth

A large offset requires the database to scan and discard the skipped rows, so page one thousand is far slower than page one.

Cursor pagination is an indexed seek to a position, so cost is the same at any depth. That is why large APIs adopt it regardless of the stability argument.

5. What It Gives Up

  • Random access. You cannot jump to page fifty; you must walk there.
  • Total page counts, since positions are not indices.
  • Backwards navigation, unless the API implements a previous cursor.
  • Bookmarkable pages, since cursors can expire or become invalid.

For interfaces with numbered page controls this is a real cost, which is why offset paging persists in user-facing listings.

6. Consuming It Correctly

  1. Treat the cursor as opaque — never parse or construct one.
  2. Stop when the cursor is absent or null, not on a short page.
  3. Cap the loop so a server that always returns a cursor cannot spin forever.
  4. Deduplicate on write anyway; tiebreaker bugs do occur.
  5. Handle expiry — a long-running traversal may need to restart.

7. Opaque Means Opaque

Cursors are usually base64-encoded and trivially decodable, which tempts clients to construct their own. Doing so couples the client to an internal implementation that can change without notice.

Encoding is not a contract; the contract is that you send back what you received.

8. Summary

Cursor pagination resumes from a marker encoding the last row's sort position, making it stable under concurrent changes and equally fast at any depth.

It trades random access and page counts for that stability. Consume it by following the server's cursor until it stops, treating the token as opaque throughout.