Proxies for Web Scraping Agencies: Per-Client Setup

Run several clients on one proxy account without mixing them up: a credential per client, a routing matrix, monthly caps and cost per accepted record.

Choose a proxy setup for each client job. A price feed may need residential exits in several countries; a partner API may need one fixed address. Keep credentials and usage separate, then price the job by accepted records, including retries.

That is the whole difference between an agency and a single scraping team: several clients share your infrastructure, and each one needs its own routing, its own budget and a bill you can defend. This guide shows how to set that up on one Node4 account.

One credential per client

On a paid Node4 account you can create up to 5 proxy credential sets under Auth & IP Access. Each set is a username and password that works on every product the account holds, and each one can carry:

• A label, such as Client: Acme, so the dashboard shows which client a credential belongs to.
• A monthly bandwidth cap, measured per calendar month in UTC. When a credential reaches its cap it stops authenticating until the month rolls over, so one client's runaway job cannot spend another client's traffic.
• Its own usage line, showing what that credential has used this calendar month.

Give each active client its own credential and put that client's username and password in that client's secret store, not in a shared .env. If a client relationship ends, delete the credential and nothing else changes.

IP whitelisting works on every product (on residential a whitelisted connection carries no username, so it is untargeted), and it is useful for your own servers, but it does not separate clients. A whitelisted address authenticates as the account, so traffic from it is counted at account level rather than against any one client's credential. Use credentials wherever per-client attribution matters.

With more active clients than credential sets, group clients by billing arrangement (for example, all fixed-fee clients on one credential) and keep per-job usage in your own logs, as described further down.

The client routing matrix

Write down, per job, which product it uses and why. This is the document a new developer reads before touching a client pipeline, and the one you show a client who asks why their bill moved.

The client names are placeholders. The choices behind each row:

• A partner that whitelists your address needs a dedicated proxy. Gateway exits change by design; a dedicated datacenter proxy keeps one address for as long as you hold it, and dedicated plans are billed per proxy rather than per gigabyte.
• Country coverage depends on the product. Node4's datacenter exits are in the United States, Italy and Spain. A job that needs Germany, Brazil or Japan goes on residential.
• A session is only for jobs that need continuity. Without a session id each new connection can draw a different exit, which suits independent page fetches. A multi-step flow gets a session id per run. A session holds its exit for up to 30 minutes from first assignment, so a run that takes longer needs a planned break.
• Rotating Datacenter and Rotating Unmetered draw from the same exits. The difference is the meter: Rotating Datacenter is billed per gigabyte, Rotating Unmetered by concurrent connections. Put a client's heavy, steady job on Unmetered and its small or bursty jobs on per-GB.

Wiring the matrix into code

Keep the matrix as data and build proxy URLs from it, so changing a client's route is a config change, not a code change. A Python version:

import os
from urllib.parse import quote

ROUTES = {
    "acme-prices-de": {
        "client": "ACME",
        "host": "gw-residential.node4.io",
        "port": 8082,
        "suffix": "-country-de",
    },
    "brightline-serp-us": {
        "client": "BRIGHTLINE",
        "host": "gw-rotating_datacenter.node4.io",
        "port": 8083,
        "suffix": "-country-us",
    },
}

def proxies_for(job: str, session: str | None = None) -> dict:
    route = ROUTES[job]
    user = os.environ[f"N4_USER_{route['client']}"] + route["suffix"]
    if session:
        user += f"-session-{session}"  # letters and digits, up to 32 characters
    password = quote(os.environ[f"N4_PASS_{route['client']}"], safe="")
    url = f"http://{user}:{password}@{route['host']}:{route['port']}"
    return {"http": url, "https": url}

# requests.get(target, proxies=proxies_for("acme-prices-de"), timeout=30)

The client's credential is chosen by the route, so a developer cannot accidentally send Acme's job through Brightline's credential. Passwords are URL-encoded because a password containing @ or : would otherwise break the URL.

Check one route before scaling it. On the rotating gateway, a country the pool cannot serve is refused with a 502 rather than sent through another country, so a typo in a country code fails loudly instead of returning plausible data from the wrong market.

Price the job by accepted records

A client pays for usable records, so cost the job the same way. Count everything the job consumed, including retries and pages your parser rejected, and divide by what you delivered.

Take a hypothetical weekly price feed on a per-GB product:

Multiply the traffic by your per-GB rate from the pricing page, add compute and storage, and divide by 37,600. That number, not the cost per request, is what goes into the client's quote and what you compare when you test a different route.

Two things move it more than anything else. Retries are paid traffic, so a target that starts challenging one request in five raises the cost of every accepted record. And page weight is billed on per-GB products, so blocking images and fonts in a browser-based job can cut traffic substantially. On Rotating Unmetered the price is set by the plan's concurrent-connection ceiling instead, so the lever is how many connections a job holds open at once.

The handoff format

Deliver the data with a short run report in the same shape every time. It answers the client's first question ("is this complete?") and gives you the numbers for the next quote.

client,job,period_start_utc,period_end_utc,pages_requested,attempts,accepted,rejected_blocked,rejected_wrong_market,rejected_parse,traffic_gb,credential_label
Acme,prices-de,2026-09-28T00:00Z,2026-10-04T23:59Z,40000,50000,37600,1200,300,900,17.5,Client: Acme

Keep the rejection columns separate. "Blocked" means the target refused the request, "wrong market" means the page loaded in the wrong country or currency, and "parse" means the data was there but your extractor missed it. Each one has a different fix, and only the first is a proxy question.

FAQ

Can each client have its own proxy username?

Yes, up to 5 credential sets per paid account, each with a label, its own usage line and an optional monthly bandwidth cap. Every set works on every product the account holds.

Do agencies need residential proxies for every client?

No. Jobs that target the US, Italy or Spain and accept datacenter traffic can run on datacenter exits. Use residential for other countries and for targets that treat hosting networks differently. Test both against the same pages and compare accepted records.

How do I stop one client's job from using another client's budget?

Put each client on its own credential and set a monthly cap on it. When the cap is reached, that credential stops authenticating until the next calendar month (UTC), and the other credentials are unaffected.

Related guides

• How many proxies you need
• Proxy pricing models compared
• Python requests with a proxy
• Rotating proxies for web scraping
• Proxies for brand protection teams
• Proxies for alt data teams

For product details see Rotating Datacenter and Rotating Unmetered and dedicated datacenter proxies.