What Is Puppeteer Cluster? – Definition and How It Works

Informational P3 what is puppeteer cluster

Learn what puppeteer-cluster manages, how its concurrency models differ, and what it does and does not solve for parallel collection.

puppeteer-cluster is a library that runs many Puppeteer tasks in parallel, managing a pool of browsers or pages and handling queueing, failure, and cleanup.

1. The Problem It Solves

Running twenty pages in parallel by hand means managing browser lifetimes, a queue, per-task failure, and cleanup of crashed instances. That is a meaningful amount of code, and it is easy to leak resources.

The library provides the pool, the queue, and the cleanup.

2. The Concurrency Models

Model	Isolation	Cost per task
Page	Shared browser, shared context	Lowest
Context	Shared browser, separate contexts	Moderate
Browser	Separate browser per task	Highest

Context-level concurrency is usually the right default: cookies and storage are isolated per task while the expensive browser process is shared.

Page-level shares state between tasks, which is faster and frequently wrong — one task's session leaks into another's.

3. The Shape

const cluster = await Cluster.launch({
  concurrency: Cluster.CONCURRENCY_CONTEXT,
  maxConcurrency: 8,
  timeout: 60_000,
});

await cluster.task(async ({ page, data: url }) => {
  await page.goto(url, { waitUntil: 'domcontentloaded' });
  await page.waitForSelector('article.product');
  return extract(page);
});

urls.forEach((url) => cluster.queue(url));
await cluster.idle();
await cluster.close();

4. What It Handles

A bounded pool, so concurrency cannot exceed the configured maximum.
A queue, so more tasks than workers is normal.
Per-task timeouts, so one stuck page does not stall the pool.
Restarting crashed browsers.
Optional per-task failure handling rather than aborting the run.

5. What It Does Not Handle

Per-host politeness. maxConcurrency bounds your machine, not your load on one destination. Eight parallel workers against one host is eight simultaneous requests.
Pacing. No delay between requests is imposed.
Selector durability.
Whether collection is permitted.

The first is the one that matters most and is most often missed: a concurrency cap is a resource control, not a courtesy.

6. Memory Is the Real Limit

Each context holds memory, and each page more. The practical maximum concurrency on a machine is usually decided by memory rather than by CPU.

Blocking images, fonts, and media reduces per-page memory substantially and often permits meaningfully higher concurrency on the same hardware.

7. Alternatives

Playwright's contexts plus a semaphore achieve the same result in a few lines, with pooling logic you can see. Crawlee provides a queue, autoscaling, and storage as well.

The library is a good fit where Puppeteer is already in use and the pooling code is the missing piece.

8. Summary

puppeteer-cluster manages a bounded pool of browsers or contexts with a queue, timeouts, and crash recovery, with context-level concurrency the sensible default.

It bounds your own resources and says nothing about load on a destination — so per-host limits and request pacing remain yours to add, and memory rather than CPU sets the practical ceiling.