Measuring Proxy Performance: Benchmark It Yourself

Connect time, TTFB, throughput and tail percentiles: a reproducible method and a script for testing any provider's claims, including ours.

Proxy marketing runs on adjectives, and adjectives do not transfer to your workload. Ours included: node4 states an uptime target of 99.9% and network capacity of 100+ Gbps, and neither figure tells you what your requests will experience from your servers, through our exits, against your targets, at your hour of day. Nobody's marketing can tell you that, because nobody else runs your workload from your network. The only performance numbers worth acting on are the ones you measure yourself, and this article is the method.

The encouraging part: a defensible proxy benchmark needs an afternoon, a shell, and a small amount of statistical discipline. The discouraging part: most published comparisons skip the discipline, which is why they disagree with each other and with everyone's production experience.

Which proxy performance numbers are worth measuring?

Four, and they answer different questions: connect time, time to first byte, total transfer time, and failure rate. Quoting one of them as the speed of a proxy is how vendor comparisons become meaningless.

Connect time. How long it takes to establish a connection to the proxy: the TCP handshake, plus TLS where the hop is encrypted. For workloads that open a fresh connection per request, which is exactly what per-request rotation implies, connect time is paid on every single request and usually dominates the total.

Time to first byte. The gap between sending the request and receiving the first byte of the response, through the whole chain: proxy processing, the proxy's own connection to the target, and the target's think time. TTFB is the metric that matches the subjective feeling of "sluggish," and the one most workloads actually care about.

Throughput. Bytes per second once data is flowing. It decides how long large transfers take and almost nothing else. An API-scraping workload pulling small JSON bodies never runs long enough per request for transfer speed to differentiate two providers. Weight it by how many of your bytes move in bulk.

Success rate. The fraction of attempts that produced a usable response, where usable is your definition, not HTTP's: the right status code, a body that parses, inside your deadline. This definition step is not pedantry. A benchmark that counts any completed transaction as success will score a challenge page as a win, and a provider quoting a success figure without stating the definition behind it has told you nothing you can compare.

Different jobs weight these four completely differently, which is the first reason generic reviews mislead: a verdict computed for someone else's workload does not rank providers for yours.

Should I compare proxies on average response time?

Never summarize any of these metrics as an average. An average blends your fast majority with your slow tail into a single number that describes neither, and it is the tail that breaks pipelines.

Use percentiles. The p50, the median, is the experience of a typical request. The p95 and p99 are the experience of the unluckiest one in twenty and one in a hundred. Two providers can post identical medians while one has a p99 many times worse, and the p99 is what sets your timeout budget, your retry volume, and whether tonight's run finishes tonight: in a job of 100,000 requests, the worst hundredth happens a thousand times. The slow tail also compounds: requests that time out get retried, and the retries land on top of whatever congestion caused the timeouts.

The related sin is survivorship. Timeouts and failures must enter the result as what they are (failed attempts), not silently vanish from it. A summary that quietly drops them is reporting the performance of the requests that happened to survive, which is a different and much rosier quantity than the performance of the proxy.

The method

Five rules turn a pile of curl output into a result you can defend:

  1. A fixed, boring target that you control. The reasons get their own section below.
  2. One variable at a time. Same client machine, same target, same request, same time period; only the proxy changes.
  3. Hundreds of samples per condition, minimum. A p99 computed from thirty requests is a coin flip wearing a lab coat. Tails need data.
  4. Spread the samples across hours. Networks have rush hours; a benchmark run entirely at a quiet time measures the quiet time.
  5. Record raw rows, summarize later. Keep every per-request measurement. A summary cannot answer the follow-up question you will have tomorrow; the raw CSV can.

The script

curl exposes exactly the timing splits we defined, via -w. This loop takes one proxy and produces one CSV row per request:

#!/usr/bin/env bash
# bench.sh — sequential probes through one proxy, one raw CSV row per request
PROXY="http://USERNAME:PASSWORD@HOST:PORT"   # the proxy under test
URL="https://bench.example.com/probe.txt"    # a small target YOU control
N=300
OUT="results-$(date +%s).csv"

echo "epoch,code,connect,tls,ttfb,total" > "$OUT"
for i in $(seq 1 "$N"); do
  curl -s -o /dev/null -x "$PROXY" --max-time 30 \
    -w "$(date +%s),%{http_code},%{time_connect},%{time_appconnect},%{time_starttransfer},%{time_total}\n" \
    "$URL" >> "$OUT"
  sleep 1
done

Reading the fields: time_connect is the TCP handshake to the proxy, time_appconnect adds the TLS negotiation, time_starttransfer is TTFB through the whole chain, and time_total is everything. Two details in that script are load-bearing. --max-time converts a hang into a data point instead of a stuck terminal. And the sleep keeps this a measurement rather than a load test; probing politely also avoids tripping rate limits that would contaminate the timing data with penalty behavior.

There is a subtlety worth stating, because getting it wrong quietly corrupts the failure count. curl writes its -w line whether or not the transfer succeeded: a timeout or a refused connection still emits a row, with http_code of 000 and zeroed timings, and only then exits non-zero. So the failures record themselves and you need no fallback arm. Adding the obvious-looking || echo "…,000,…" produces two rows for every failed probe, roughly doubling the failure count and leaving the file with more rows than you made requests. Let curl speak once per request.

Summarize with percentiles, and count the failures separately:

# p50 / p95 / p99 of total time, successful rows only
awk -F, 'NR>1 && $2=="200" {print $6}' "$OUT" | sort -n | awk '
  {v[NR]=$1}
  END { printf "n=%d p50=%.3f p95=%.3f p99=%.3f\n",
        NR, v[int(NR*.50)+1], v[int(NR*.95)+1], v[int(NR*.99)+1] }'

# failures are part of the result, not noise
awk -F, 'NR>1 && $2!="200"' "$OUT" | wc -l

For throughput, swap the probe file for a larger one and read %{speed_download}; for a concurrency view, run several copies of the loop in parallel, but change one thing at a time, and label each run with what it was.

Two subtleties that quietly skew results

DNS. Each fresh curl invocation may pay a DNS lookup, and resolver caching means the first probe of a batch pays more of it than the rest. Decide whether resolution belongs inside your measurement (it genuinely is part of a cold request's cost), and if you want it out, pin the hostname with --resolve so every probe skips the lookup identically. Whichever you choose, choose it for every condition you compare.

The machine doing the measuring. A loaded laptop on Wi-Fi adds jitter that lands in your CSV indistinguishable from network behavior, and CPU contention inflates the TLS timings specifically. Benchmark from an idle, wired machine, ideally the same class of host your production traffic will leave from, for the same reason the target region should match production.

What should a proxy benchmark point at?

Something you control, or something deliberately trivial to serve. Anything else and the numbers describe the destination rather than the proxy.

The commonest error in amateur proxy benchmarks is pointing them at a big public website. The numbers that come back are dominated by things that have nothing to do with the proxy: the site's CDN routing, its caching mood, its geographic load balancing, and, worst of all, its bot defenses, which may deliberately serve classified visitors slowly. Route two providers at such a site and you are comparing how a third party treats them, which is a real question but not the one you asked.

Host the probe yourself: a small static file on a VPS or a storage bucket in a region you choose. Then take a direct baseline: the same probe loop with no proxy, from the same machine, interleaved in time with the proxied runs. The proxy's true contribution is roughly the gap between the two distributions, and without the baseline you cannot distinguish a slow proxy from a slow patch of internet between you and the test region.

Geography deserves an explicit decision rather than an accidental one. A provider whose exits sit near your test target will beat one whose exits are far from it, regardless of either network's quality. If your production targets are in one region, put the probe there and let geography count; that is realism. If you are trying to isolate network quality, test against probes in several regions. Either choice is fine; discovering afterwards that geography decided your result is not.

Why do the numbers change when my client reuses connections?

Because a reused connection pays the TCP and TLS setup once instead of on every request. Warm and cold are two different benchmarks, and averaging them together produces a figure that describes neither.

A client that keeps its connection to the proxy open pays the TCP and TLS setup once and reuses the channel; every request after the first rides warm. The script above launches a fresh curl per request, so every request pays full price: all cold. Neither number is wrong; they are honest measurements of two different workloads.

Match the mode to your job. Long browser-automation sessions and sticky-session scraping live mostly warm. Per-request rotation lives permanently cold, because a fresh exit means a fresh connection. That is why connect time, not throughput, decides how rotation feels in practice. If you run both kinds of workload, measure both modes and label the results, because a warm number quoted against a cold number makes any comparison meaningless. This, incidentally, is a polite question to ask of any published benchmark, since few of them say which they measured.

How do I compare two proxy providers fairly?

The same discipline, with symmetry added:

A quiet word on what "winning" means: for most scraping pipelines, the provider with the modestly slower median and the cleaner tail is the better buy, because retries and timeouts cost more wall-clock time than medians ever save. Your measured failure fraction and p99 will tell you; the brochure will not.

What to do with the numbers

Set your client timeouts from your measured p95 or p99, with margin, not from a vendor page or a colleague's anecdote from a different continent. Set your retry budget from the measured failure fraction. When a re-run shows the failures changing character, the error code taxonomy sorts proxy-side from target-side before you blame anyone. Feed the connect-time figure into the pool-sizing arithmetic, since per-request duration is what couples concurrency to throughput. A benchmark is a snapshot, though, and the failures worth catching are the ones that appear between runs: turn the same measurements into a standing dashboard and you get alerts instead of archaeology, which is what proxy analytics covers. Then keep the raw CSVs and re-run the same benchmark monthly and after any change on your side: a trend across months is worth more than any single run, and if the trend bends the wrong way, slow proxy speeds works through the likely causes on both your side and ours.

Finally, apply this standard to us exactly as sternly as to anyone else. We publish an uptime target and a capacity figure; we do not publish latency numbers, because whatever we measured from our racks would not be what you get from yours, and printing it anyway is how marketing pages end up contradicting their own users' stopwatches. Run the script instead. The free tier provisions 3 free shared proxies with 1 GB/month bandwidth and 10 concurrent threads: enough to point this exact benchmark at our network before any money moves. If your measurements say we fit your workload, they will keep saying so on re-runs; if they say we do not, believe them over anything we write here.