Puppeteer Rotating Proxy: Browser, Context or Gateway

One Chromium launch is one exit IP. How to rotate in Puppeteer per browser, per context or at the gateway, and how to tell which your job needs.

--proxy-server is a launch argument, and that single fact is the whole shape of proxy rotation in Puppeteer. Chromium reads the flag once, when the process starts, and every page opened from that browser shares the exit it names. There is no per-request switch, no page.setProxy(), and nothing in the framework that will rotate for you. A job that needs more than one exit address therefore has exactly three places it can get them: a new browser process, a new browser context inside the running process, or a gateway that rotates behind one fixed address.

This guide is the rotation half of the Puppeteer story, with runnable code for each of the three and what each costs. The other half, getting credentials into Chromium at all (the flag drops them; page.authenticate is the answer) and the DNS and WebRTC side channels, is in proxy authentication in Puppeteer and Playwright and is not repeated here. If a 407 or an ERR_INVALID_AUTH_CREDENTIALS is what brought you, start there.

The base case, and the check that comes before rotation

One browser, one proxy, one verification:

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  args: ["--proxy-server=http://HOST:PORT"],
});
const page = await browser.newPage();
await page.authenticate({ username: "USERNAME", password: "PASSWORD" });

await page.goto("https://api.ipify.org/");
console.log(await page.$eval("body", (el) => el.innerText.trim()));
await browser.close();

Print the address before doing anything else, because a flag that did not take does not throw: Chromium simply browses from your own machine, and a job can run to completion that way. Every rotation mechanism below is verified the same way, by asking two workers for their exit and asserting the answers differ.

Rotation per browser: a worker pool over a proxy list

The oldest pattern, and for a small static list still a reasonable one. Each worker launches its own Chromium with its own --proxy-server, pulls URLs from a shared queue, and closes when the queue is empty:

import puppeteer from "puppeteer";

const PROXIES = [
  "http://203.0.113.10:8080",
  "http://203.0.113.11:8080",
  "http://203.0.113.12:8080",
];
const queue = [...urls];

async function worker(proxy) {
  const browser = await puppeteer.launch({
    args: [`--proxy-server=${proxy}`],
  });
  try {
    while (queue.length) {
      const url = queue.shift();
      const page = await browser.newPage();
      await page.authenticate({ username: "USERNAME", password: "PASSWORD" });
      await page.goto(url, { waitUntil: "domcontentloaded" });
      await handle(page);
      await page.close();
    }
  } finally {
    await browser.close();
  }
}

await Promise.all(PROXIES.map(worker));

The finally is not optional. A crashed worker that leaves its Chromium running is a common source of memory growth in long scraping jobs, and the leak is invisible until the box swaps.

What this costs is a full browser per exit. Chromium is a process tree measured in hundreds of megabytes, so three exits is three of everything and thirty exits is a machine. The list is also yours to keep healthy: an address that stops answering stays in PROXIES until you remove it, and there is no signal in this loop that would tell you. Whitelisting the machine's address on the Auth & IP Access page removes the authenticate call from every page, which matters more here than anywhere, because it is called once per URL. The scope depends on the product, and IP whitelisting works on every product: on static datacenter proxies like this list, nothing is lost; on the rotating gateways a whitelisted machine can still send the username with an empty password to keep country and session targeting; on the residential gateway a whitelisted connection sends no username at all, so it is untargeted and rotates freely, and anything that pins a country or session keeps its credentials.

If the list came with credentials baked into each URL and you would rather not thread them through every page, proxy-chain turns an authenticated upstream into a local credential-free one:

import { anonymizeProxy, closeAnonymizedProxy } from "proxy-chain";

const local = await anonymizeProxy("http://USERNAME:PASSWORD@203.0.113.10:8080");
// local is something like http://127.0.0.1:41233; pass it to --proxy-server
// ...
await closeAnonymizedProxy(local, true);

One forwarder per upstream, closed when the worker closes. It is a convenience, not a rotation mechanism; the exit is still fixed for the browser's lifetime.

Rotation per context: many exits in one process

Puppeteer 22 and later expose the proxy on the browser context, which is an isolated profile (cookies, storage, cache) inside a running browser. Older releases spell it createIncognitoBrowserContext with the same options. One Chromium hosts as many contexts as memory allows, and each carries its own exit:

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  // A placeholder at launch; the contexts override it.
  args: ["--proxy-server=http://per-context"],
});

async function contextFor(proxy) {
  const context = await browser.createBrowserContext({
    proxyServer: proxy,
    proxyBypassList: ["localhost"],
  });
  const page = await context.newPage();
  await page.authenticate({ username: "USERNAME", password: "PASSWORD" });
  return { context, page };
}

const a = await contextFor("http://203.0.113.10:8080");
const b = await contextFor("http://203.0.113.11:8080");

await a.page.goto("https://api.ipify.org/");
await b.page.goto("https://api.ipify.org/");
console.log(await a.page.$eval("body", (el) => el.innerText));
console.log(await b.page.$eval("body", (el) => el.innerText));

await a.context.close();
await b.context.close();
await browser.close();

Note what stays per page. The context sets the address; credentials still arrive through page.authenticate on every page inside it, and a page that skips the call gets 407s while its siblings work. Note also the placeholder in the launch arguments. Chromium on Windows has ignored context-level proxies unless the browser itself was launched with one (Playwright documented the placeholder-at-launch convention for exactly this case), and the failure is silent: both contexts print your own address. The help article linked above explains the caveat in full; the two-line assertion at the end of the example is how you find out whether your build has it.

Contexts are the right shape for a worker pool because they are cheap to destroy and recreate. When a worker's identity goes bad, closing its context and opening a fresh one with a different proxy is a rotation that costs a few milliseconds instead of a browser launch. That property is what the last two sections build on.

Rotation at the gateway: one address, a pool behind it

The mechanism that removes the list entirely. Point every browser or context at gw-rotating_datacenter.node4.io and each new HTTPS tunnel through it gets its own exit from the pool; the pool, its health and its churn are behind the hostname. Nothing in the Puppeteer code changes when the pool does.

The catch is that a browser is not a request-at-a-time client. Loading one page opens several connections, each HTTPS connection a tunnel holding its own exit, and keeps them alive for the next page, so a single document can come from one exit and its scripts and images from others, and the next navigation may reuse any of them. Most targets do not notice. Targets that tie a session to an address do, and the symptom is a login that succeeds and a following page that treats you as a stranger.

The fix is to pin a session per context and rotate by replacing the context. The session id rides in the username as a hyphenated segment, so it belongs in page.authenticate, not in the proxy address:

import { randomBytes } from "node:crypto";

const GATEWAY = "http://gw-residential.node4.io:8082";

const browser = await puppeteer.launch({
  args: [`--proxy-server=${GATEWAY}`],
});

async function sessionContext() {
  const id = randomBytes(6).toString("hex");
  const context = await browser.createBrowserContext();
  const page = await context.newPage();
  await page.authenticate({
    username: `USERNAME-session-${id}`,
    password: "PASSWORD",
  });
  return { id, context, page };
}

Every page loaded in that context now shares one exit for the session's lifetime, and a new identity is one sessionContext() call away. Pinning is a first-class feature of the residential gateway. On the rotating gateways it depends on the gateway's session store: when that store is unreachable a pinned request is refused with a 503 rather than served from a rotating exit, and every rotating response carries an X-Node4-Sticky header (held, opened, none, unavailable), where none on a request you meant to pin means no session segment arrived (a misspelled segment key is refused with a 407 instead). Run the two-context assertion before a job depends on the pin. Add a country segment (USERNAME-country-de-session-${id}) to hold the exit in one country as well; the grammar is in residential country and city targeting.

Whitelisting changes the arithmetic here in one way worth knowing before you choose it: a whitelisted residential connection sends no username, so it is untargeted: it cannot carry a session or a country and rotates freely across the pool. If the job needs either, keep the credentials.

Rotate on evidence, not on a timer

Rotating every N requests is a common tutorial pattern, and it is the wrong trigger. It burns identities that were working and keeps ones that were already blocked for the rest of their turn. The better trigger is the target's own signal: a 403, a 429, a challenge page where the content should be. Watch the response, and replace the context when the verdict arrives:

async function fetchWithRotation(url, attempts = 3) {
  for (let i = 0; i < attempts; i++) {
    const s = await sessionContext();
    try {
      const res = await s.page.goto(url, { waitUntil: "domcontentloaded" });
      const status = res?.status() ?? 0;
      const challenged = await s.page.$("form[action*='captcha'], #challenge-form");
      if (status < 400 && !challenged) return await s.page.content();
      console.warn(`session ${s.id}: ${status}${challenged ? " + challenge" : ""}, rotating`);
    } finally {
      await s.context.close();
    }
  }
  throw new Error(`${url}: blocked on ${attempts} sessions`);
}

The selector in the middle is a stand-in; every target renders its interstitial differently, and the only reliable check is one written against the specific page you are crawling. What the loop gets right is the accounting: a context lives exactly as long as the target accepts it, and three consecutive blocks on three separate exits stop the job instead of spending the rest of the pool on a page that will not load.

Three blocks in a row is also the moment to stop rotating and start diagnosing, because the address was probably not the trigger. A headless Chromium announces itself through navigator.webdriver, missing plugins, a TLS handshake that does not match its user agent and timing no human produces, and a fresh exit inherits that verdict within a page or two. Address reputation and fingerprint fail in different ways and need separate fixes; why your IP is getting blocked tells them apart.

Where datacenter exits lose

Rotation solves per-address rate limits. It does not solve classification, and a defended target classifies. It looks up the network an address belongs to and treats the entire hosting range as automation, so a datacenter pool rotates through a thousand addresses that all carry the same verdict. A job that ran cleanly against a lenient site and gets nothing but challenges from a serious one, with the fingerprint unchanged, has usually hit this.

The answer at that point is a residential exit, whose address belongs to a consumer network, driven with credentials so the session segment can hold one identity through a login or a checkout. It is the more expensive product, and buying it before the evidence arrives is the common overspend; start on the rotating gateway, which is datacenter and metered on its Rotating Datacenter tier, and move only the jobs that are classified.

The question underneath both is whether a browser is needed at all. Chromium fetches every stylesheet, script, image and font a page references, and through a metered plan every one of those is on the bill; the resource-blocking recipe in the help article cuts it, but a plain HTTP fetch is still an order of magnitude lighter. When the data is in the HTML or behind a JSON endpoint, a Scrapy spider through the same gateway rotates more cleanly and costs less per page, and proxies for Scrapy covers that product choice. Keep Puppeteer for the pages that only exist after JavaScript runs.

Which mechanism for which job

JobMechanismWhy
A few hundred pages, a handful of static proxiesOne browser per proxyNothing to build; the list is small enough to watch by hand
A pool of workers with distinct identitiesOne context per workerCheap to create, cheap to replace, one process
Unattended crawl, no list to maintainGateway, context per sessionRotation and pool health are behind the hostname
Login-then-browse flowsGateway with a pinned session per contextOne exit for the whole flow; rotate by replacing the context
Defended target, datacenter blockedResidential gateway, credentials, session per contextClassification, not rate, is the problem

The errors specific to rotation

Before touching any of it, run curl -x http://HOST:PORT https://api.ipify.org/ with the same credentials. If curl fails, Puppeteer was never the problem, and proxy connection refused or timing out is the next page to read. Plans, and which gateways each includes, are on the pricing page. If the question is which proxy type to point this at rather than how to wire it, proxies for Puppeteer and Playwright: choosing a type works through that choice, and proxies for browser automation is the same decision with the products and their prices beside it.