Proxy Analytics: Track Usage, Latency and Failed Requests

What to record so proxy problems show up: Node4's request and bandwidth figures, your client's latency and content checks, and how to join the two.

Track two things separately: how much proxy traffic you send, and whether each request returns the data your job needs. Node4 reports requests and bandwidth. Your client must record latency, target status and content checks, because HTTPS stays encrypted through the proxy and those results never reach Node4. Join the two over the same time window and the same traffic, and you can tell a slow or refused proxy from a target that is answering with the wrong page.

Table of Contents

What Should You Actually Track In Proxy Analytics?

Each metric below has one source that can actually see it. Putting a metric on the wrong side of that line is how a dashboard ends up showing a healthy proxy while the dataset fills with challenge pages.

Two columns keep scope straight. Window and grouping is the time bucket and the slice the figure covers. Counted over is the population a rate or percentile is computed from; a total has none.

MetricCollected byWindow and groupingCounted overDecision it supports
Bandwidth (bytes up and down)Node4Hour, day or month for the whole account; lifetime per proxyA totalAllowance, top-ups, cost per job
Proxied requestsNode4Same as bandwidthA totalVolume trend; compare against your own attempt count
Response latency, p50 / p95 / p99Your clientPer target, per windowAttempts that got a responseTimeouts and concurrency
Time to failureYour clientPer target, per windowAttempts that ended without a responseWhether timeouts are set right, and how long failures hold workers
Target status codesYour clientPer target, per windowAttempts that got a responseWhether the target is refusing or erroring
Accepted resultsYour client (content check)Per target, per windowAll attemptsWhether the job is producing usable data
Retries per accepted resultYour clientPer job and target, per windowAccepted resultsHidden retry load and cost
Proxy errors (407, connect failures, timeouts)Your clientPer proxy label, per windowAll attemptsCredentials, network path or capacity
Bytes per accepted resultBoth, joinedOne window, with the same traffic on both sidesAccepted resultsWhether spend is buying data or failures

Two of these need care. Proxied requests on Node4 are counted per proxied connection: one for each HTTPS tunnel (a CONNECT), each SOCKS5 connection, and each plain-HTTP request. A client that sends many HTTPS requests over one kept-alive tunnel will log more requests than Node4 counts, so compare trends, not raw totals.

Latency needs percentiles rather than an average. The median describes a typical request; p95 and p99 describe the slow tail that sets your timeout and retry budget. If your p50 is 200 milliseconds but your p99 sits at 8 seconds, one request in a hundred is holding a worker for 8 seconds, and an average will not show it. A timeout has no response, but it still has an elapsed time: roughly the timeout you set, or less for a refused connection. Keep it as a failed attempt with its duration, report it as time to failure, and keep it out of the response-latency percentiles. Otherwise one of two things goes wrong: mixed in, failures pile up at your timeout value and distort the tail; dropped, the percentiles describe only the requests that survived. Read response latency next to the failure share. Measuring proxy performance has the full benchmarking method.

Accepted results is the metric most dashboards lack. A 200 only says the target answered. Define what a usable answer looks like for each target (an expected element, a minimum body size, the absence of a login or challenge marker) and count it separately.

How Do You Collect Reliable Proxy Telemetry?

Write one structured log row per attempt in your client, including attempts that time out or fail before a response. Rows like these are enough to compute every client-side metric in the table above. The values are illustrative:

{"ts":"2026-10-18T09:14:03.512Z","job":"price-check","target":"shop.example","proxy":"dc-us-0042","attempt":2,"duration_ms":1840,"proxy_error":null,"status":200,"bytes":48211,"accepted":false,"reject_reason":"challenge_page"}
{"ts":"2026-10-18T09:14:05.020Z","job":"price-check","target":"shop.example","proxy":"dc-us-0042","attempt":3,"duration_ms":30000,"proxy_error":"read_timeout","status":null,"bytes":0,"accepted":false,"reject_reason":null}
  • target is the host name, not the full URL. Paths and query strings can carry account identifiers or session tokens, and the host is what you group by.
  • proxy is your own label for the exit or product you used, so rows can be joined with Node4's per-proxy usage. Never log the proxy password or a full proxy URL that contains it.
  • attempt numbers the retries for one logical fetch, which is what makes retry load measurable.
  • duration_ms is the elapsed time of the attempt, whether or not a response arrived.
  • proxy_error records attempts that ended without a target response, with the stage where they ended: proxy_407, connect_refused or connect_timeout before the tunnel opened, or read_timeout after the request was sent. Only the connect-stage errors show the request never reached the target. After a read timeout you cannot tell whether the target received the request, or even acted on it, so treat a retry of anything that changes state with care. Keep all of these out of status so a proxy problem is not counted as a target refusal.
  • bytes is what your client received for that attempt. It will not match Node4's bandwidth for the same traffic, which also includes upload and protocol overhead such as headers and TLS, but it is the only byte count you can split by target.
  • accepted and reject_reason come from the content check, not the status code.

Ship the rows to whatever you already run for logs or metrics. The format matters less than having every attempt in one place with the same fields.

What Do You Do When Proxy Analytics Show Trouble?

A worked example of why the content check matters. A price-monitoring job runs overnight against one retailer, and it is the only traffic on its Node4 account during those hours, so the account's hourly figures are this job's figures. In the morning, the share of 200 responses for that target has gone up, latency is flat, and Node4's bandwidth for the run is about the same as the night before. Read from status codes alone, the run looks healthier than usual. But the accepted-results count for that target has fallen by half, and the reject_reason column is mostly challenge_page.

The target is answering, with a page your job cannot use. Work through it in order:

  1. Confirm the scope. Filter accepted results by target. If only one target fell, the proxy network as a whole is unlikely to be the cause; if every target fell at once, look at your client and the proxy path first.
  2. Read the bodies. Open a sample of rejected responses. A challenge page, a login wall, a consent page and a "no results" page each point to a different cause, and only the body tells you which.
  3. Check what changed on your side. Request rate, concurrency, headers, cookies or session state, and any account the job logs into. A change in your own deploy window is a more common cause than a change in the network.
  4. Check the target. A redesign or a new bot-management rule can change what the same request receives.
  5. Check the cost. With bandwidth flat and accepted results halved, bytes per accepted result has roughly doubled, because the job is paying for bandwidth that returned nothing usable. That is the number to put in front of whoever owns the budget. It holds here because the job ran alone; on an account shared with other work, see the attribution notes in the next sections.

Only after those checks is it worth changing how the job presents itself, its request rate or its proxy type. The symptom alone does not say which fix is right, and a status-code dashboard would have reported the night as a success.

How Do You Set Alert Thresholds That Don't Cry Wolf?

A fixed threshold such as "alert above 2 seconds" fires constantly where normal load varies by hour and target. Compare against your own recent baseline instead, chosen for the workload:

  • Pick a window that covers the job's normal cycle. A job that runs nightly needs several nights of history; one that runs continuously may need a week to include weekday and weekend patterns. There is no universal number of hours; re-check the baseline after any change to the workload, the target or the proxy product.
  • Require enough samples. A p99 from thirty requests is noise. Alert on percentiles only for targets with enough attempts in the window, and on counts for the rest.
  • Count missing responses. Timeouts and connect failures never enter a response-latency percentile, so the percentile can improve while failures climb. Alert on failure share and time to failure alongside latency.
  • Alert on sustained deviation. Require the change to persist across more than one evaluation window before paging anyone.
  • Look for corroboration. Rising latency with a falling accepted-results count is a problem. Rising latency alone may just be a slow target.
  • Watch retry load. Retries can hold the accepted count steady while traffic and cost climb. Chart retries per accepted result next to the success rate.

Does Node4 Cover the Full Proxy Analytics Checklist?

Partly. Node4 covers the volume half, at account, proxy and login level, not per target. The dashboard's Analytics page shows bandwidth and request history, including per proxy, and consumption against your allowance is on the Dashboard. On plans with a bandwidth allowance, the Usage Alerts tab in Account Settings can send an email or call a webhook at 80% and at 100% of it. It is included with every plan rather than sold as an add-on. Analytics is one part of a larger console; the proxy dashboard management checklist covers the rest, from session control to billing.

The same data is in the REST API under the read:usage scope:

EndpointReturns
GET /usageThis month's and all-time upload, download and request totals, plus recent usage log entries, each with a proxyId and a reporting period
GET /usage/hourly?hours=48Hourly entries for the whole account: hour, bandwidthBytes, requests (up to 168 hours)
GET /usage/daily?days=30The same per UTC day (up to 90 days)
GET /usage/by-proxyLifetime uploadBytes, downloadBytes, requests per proxy, by host and port; traffic that cannot be tied to one proxy is an account-level row
GET /usage/analytics?months=6Per-month totals, oldest first
GET /sub-users/{id}/usageOne login's bandwidth for the current UTC month; no hourly or daily series

An hour or day that carried no traffic is absent from the hourly and daily series, so treat a missing entry as zero. Sum the hourly entries that share an hour value rather than assuming one entry per hour:

curl -s -H "Authorization: Bearer $NODE4_API_KEY" \
  "https://node4.io/api/v1/usage/hourly?hours=48"

To join the two halves, the numerator and the denominator must cover the same traffic: the same account or login, the same proxies, the same targets and the same hours. The hourly series is account-wide, so dividing it by one target's accepted results only works when that target's job is the only traffic in those hours. A worked version, with illustrative numbers: the price-check job runs alone from 01:00 to 04:00 UTC. Once those hours have closed and stopped changing, the hourly entries for 01:00, 02:00 and 03:00 sum to 1.8 GB, and the client log shows 9,000 accepted results with a timestamp in the same three hours. Bytes per accepted result is about 200 KB. Count accepted results by the hour the attempt ran, and use whole UTC hours, because that is how the series is bucketed.

When several jobs share the account, isolate them before dividing. Give the job its own sub-user login and read that login's monthly figure before the run and again a few minutes after it ends (usage reaches the counter in short batches), or run it on proxies no other job uses and read their usage. Otherwise, split by target with your client's bytes field and treat the result as an estimate of each target's share, not as Node4's billed figure. Getting started with the Node4 API covers creating a scoped key, and the API reference lists every usage field.

The other half is yours to instrument. A target's status codes, challenge pages and per-request latency travel inside an HTTPS tunnel that the proxy relays without reading, so they never reach Node4's reporting. Requests refused at the proxy with a 407 never reach a target, so count authentication failures in your client as well.

How Should You Handle Data Privacy in Proxy Analytics?

Your client logs can say a lot about what you collect and from where, so give them the access rules you would give any operational data. Most people need the aggregated success and latency views, not the raw rows. Keep credentials out of the rows entirely, log host names rather than full URLs unless a path is genuinely needed, and decide on a retention period from your own investigation, privacy and legal needs rather than a default.

Node4 keeps its own connection records on its side of the boundary. What they contain and how long they are kept is set out in the privacy policy.

Ready to Put This Scorecard to Work?

Start with one job against one target, run while nothing else uses the account. Add the attempt log above to its client, pull GET /usage/hourly for the same whole hours, and compute bytes per accepted result for that target. Add targets once the numbers make sense, using a separate login or separate proxies for each one you want to price. If the job runs on Node4's dedicated datacenter proxies, per-proxy usage can be matched to the proxy label in your rows. On the residential gateway and the rotating gateways, the exit changes underneath one login, so use the login, not the exit, as the label.