Proxies for Web Scraping: Choosing by Proxy Type
Pick a scraping proxy by what blocks you: rotating datacenter for rate limits, residential for hosting-range blocks, static IPs for fixed-address access.
Choose the proxy type by what is stopping your scraper. If the site rate-limits one address, rotating datacenter spreads requests across many. If it refuses hosting ranges outright or shows different content by country, use residential. If a partner API only accepts a fixed address, use a dedicated static IP. This guide compares proxy types, using Node4's products as the worked example; Node4 publishes it and sells each one. It does not rank vendors.
Find out what is blocking you first
Run the same 200 requests three ways before you buy anything: direct from your server, through one datacenter address, and through a residential exit. The pattern tells you the type.
- Direct works, then fails after a burst. That is a per-address rate limit. More addresses fix it, and datacenter ones are cheapest.
- Datacenter fails from the first request, residential succeeds. The site classifies hosting ranges. Only residential gets through.
- Both succeed, but prices or currency differ from what a local visitor sees. That is a location problem. You need an exit in the right country.
- Both fail. The problem is probably not the address. Check headers, cookies, JavaScript rendering and request rate before paying for a different proxy.
For example, suppose you request a UK product page and the scraper stores a price in USD next to a US delivery address. That row is from the wrong market and should be rejected, whatever the HTTP status was.
Proxy types for scraping, side by side
| Type | Fixes | Billing unit | Exit locations | Session behavior | Main limit |
|---|---|---|---|---|---|
| Rotating Datacenter | Per-address rate limits | Per GB | US, Italy, Spain (owned) | New exit per HTTPS tunnel, SOCKS5 connection or plain-HTTP request; session id holds one for up to 30 minutes | Hosting ranges are refused by some sites |
| Rotating Unmetered | The same, at high data volume | Concurrent connections | US, Italy, Spain (owned) | Same pool and behavior as Rotating Datacenter | New connections are refused while every one your size allows is open |
| Residential | Hosting-range blocks, country-specific content | Per GB | 170+ countries (partner network) | Exit per tunnel or request; sessions best effort, up to 30 min | Costs more per GB; speed depends on the home connection |
| Dedicated datacenter | APIs that accept only registered addresses, a stable identity | Per proxy per month, no data cap | US, Italy, Spain (owned) | Same address every time | One address carries all your reputation |
| Shared datacenter | Light jobs and testing | Per proxy per month, metered | US, Italy, Spain (owned) | Same address, shared with a few other accounts | Other tenants affect the address's history |
Rotating datacenter: for rate limits on sites that accept hosting ranges
The gateway picks an exit for each HTTPS tunnel, each SOCKS5 connection and each plain-HTTP request. HTTPS requests that a keep-alive client sends through one open tunnel keep that tunnel's exit. To spread HTTPS load, let your HTTP client open new connections rather than reusing one for the whole crawl. For pagination or a login, add a session id to the username; on the rotating datacenter gateway it holds one exit for up to 30 minutes from first assignment, so a longer crawl should start a new session id partway through. Rotating proxies for web scraping has runnable examples.
The two rotating products draw the same exits. Pay by the gigabyte on Rotating Datacenter if the job is small or irregular. Pay by concurrent connections on Rotating Unmetered if you move a lot of data through a steady number of connections, since that plan has no data cap and no per-GB charge. Scrapy users can start from the Scrapy setup.
Residential: when hosting ranges are refused or location changes the page
Residential exits are consumer connections in 170+ countries, sourced from a partner network rather than owned by Node4. Put the country, state or city in the username, and an unservable location is refused rather than silently swapped. A residential session is best effort for up to 30 minutes and can end sooner if its device goes offline, so code for the exit changing. Residential costs more per gigabyte than datacenter ($0.99/GB on the 10 GB plan, against $0.59/GB on the smallest Rotating Datacenter plan), and each exit's speed depends on the home connection behind it, so move a target here only after the test above shows datacenter failing. Plan sizes are on residential plan prices.
Static datacenter: for fixed-address access and stable identity
Some targets are not adversarial at all: a partner API that only accepts registered addresses, or a dashboard that ties a session to one IP. A dedicated proxy is one address on Node4-owned blocks, yours alone, billed per proxy per month with no data cap (dedicated pricing). Shared proxies are the same blocks with a few accounts per address and a monthly data allowance (shared pricing). They are fine for testing a scraper, and weaker for production because another tenant's behavior affects the address.
Node4's free plan is 3 free shared proxies, 1 GB a month, no card required, HTTP(S) only. The free plan has no end date; if your free proxies carry no traffic for 30 days we pause them, and you can reactivate a fresh set from your dashboard in one click. It is enough to run the three-way test above on the datacenter side. It cannot reach the rotating or residential gateways.
The decision
Start with rotating datacenter if the site accepts hosting ranges and the only problem is volume per address; choose the per-GB or per-connection meter by your monthly data. Move to residential when datacenter fails the test or the page changes by country, and only for the targets that need it. Use a dedicated static IP when a target needs to recognize you. The same choice is worked through for specific jobs in Amazon scraping, price monitoring, ad verification, SEO rank tracking, TikTok and sneaker checkouts.