Proxy Analytics: Track Usage, Latency and Failed Requests
What to record so proxy problems show up: Node4's request and bandwidth figures, your client's latency and content checks, and how to join the two.
Track two things separately: how much proxy traffic you send, and whether each request returns the data your job needs. Node4 reports requests and bandwidth. Your client must record latency, target status and content checks, because HTTPS stays encrypted through the proxy and those results never reach Node4. Join the two over the same time window and the same traffic, and you can tell a slow or refused proxy from a target that is answering with the wrong page.
Table of Contents
- What Should You Actually Track In Proxy Analytics?
- How Do You Collect Reliable Proxy Telemetry?
- What Do You Do When Proxy Analytics Show Trouble?
- How Do You Set Alert Thresholds That Don't Cry Wolf?
- Does Node4 Cover the Full Proxy Analytics Checklist?
- How Should You Handle Data Privacy in Proxy Analytics?
- Ready to Put This Scorecard to Work?
What Should You Actually Track In Proxy Analytics?
Each metric below has one source that can actually see it. Putting a metric on the wrong side of that line is how a dashboard ends up showing a healthy proxy while the dataset fills with challenge pages.
Two columns keep scope straight. Window and grouping is the time bucket and the slice the figure covers. Counted over is the population a rate or percentile is computed from; a total has none.
| Metric | Collected by | Window and grouping | Counted over | Decision it supports |
|---|---|---|---|---|
| Bandwidth (bytes up and down) | Node4 | Hour, day or month for the whole account; lifetime per proxy | A total | Allowance, top-ups, cost per job |
| Proxied requests | Node4 | Same as bandwidth | A total | Volume trend; compare against your own attempt count |
| Response latency, p50 / p95 / p99 | Your client | Per target, per window | Attempts that got a response | Timeouts and concurrency |
| Time to failure | Your client | Per target, per window | Attempts that ended without a response | Whether timeouts are set right, and how long failures hold workers |
| Target status codes | Your client | Per target, per window | Attempts that got a response | Whether the target is refusing or erroring |
| Accepted results | Your client (content check) | Per target, per window | All attempts | Whether the job is producing usable data |
| Retries per accepted result | Your client | Per job and target, per window | Accepted results | Hidden retry load and cost |
| Proxy errors (407, connect failures, timeouts) | Your client | Per proxy label, per window | All attempts | Credentials, network path or capacity |
| Bytes per accepted result | Both, joined | One window, with the same traffic on both sides | Accepted results | Whether spend is buying data or failures |
Two of these need care. Proxied requests on Node4 are counted per proxied connection: one for each HTTPS tunnel (a CONNECT), each SOCKS5 connection, and each plain-HTTP request. A client that sends many HTTPS requests over one kept-alive tunnel will log more requests than Node4 counts, so compare trends, not raw totals.
Latency needs percentiles rather than an average. The median describes a typical request; p95 and p99 describe the slow tail that sets your timeout and retry budget. If your p50 is 200 milliseconds but your p99 sits at 8 seconds, one request in a hundred is holding a worker for 8 seconds, and an average will not show it. A timeout has no response, but it still has an elapsed time: roughly the timeout you set, or less for a refused connection. Keep it as a failed attempt with its duration, report it as time to failure, and keep it out of the response-latency percentiles. Otherwise one of two things goes wrong: mixed in, failures pile up at your timeout value and distort the tail; dropped, the percentiles describe only the requests that survived. Read response latency next to the failure share. Measuring proxy performance has the full benchmarking method.
Accepted results is the metric most dashboards lack. A 200 only says the target answered. Define what a usable answer looks like for each target (an expected element, a minimum body size, the absence of a login or challenge marker) and count it separately.
How Do You Collect Reliable Proxy Telemetry?
Write one structured log row per attempt in your client, including attempts that time out or fail before a response. Rows like these are enough to compute every client-side metric in the table above. The values are illustrative:
{"ts":"2026-10-18T09:14:03.512Z","job":"price-check","target":"shop.example","proxy":"dc-us-0042","attempt":2,"duration_ms":1840,"proxy_error":null,"status":200,"bytes":48211,"accepted":false,"reject_reason":"challenge_page"}
{"ts":"2026-10-18T09:14:05.020Z","job":"price-check","target":"shop.example","proxy":"dc-us-0042","attempt":3,"duration_ms":30000,"proxy_error":"read_timeout","status":null,"bytes":0,"accepted":false,"reject_reason":null}targetis the host name, not the full URL. Paths and query strings can carry account identifiers or session tokens, and the host is what you group by.proxyis your own label for the exit or product you used, so rows can be joined with Node4's per-proxy usage. Never log the proxy password or a full proxy URL that contains it.attemptnumbers the retries for one logical fetch, which is what makes retry load measurable.duration_msis the elapsed time of the attempt, whether or not a response arrived.proxy_errorrecords attempts that ended without a target response, with the stage where they ended:proxy_407,connect_refusedorconnect_timeoutbefore the tunnel opened, orread_timeoutafter the request was sent. Only the connect-stage errors show the request never reached the target. After a read timeout you cannot tell whether the target received the request, or even acted on it, so treat a retry of anything that changes state with care. Keep all of these out ofstatusso a proxy problem is not counted as a target refusal.bytesis what your client received for that attempt. It will not match Node4's bandwidth for the same traffic, which also includes upload and protocol overhead such as headers and TLS, but it is the only byte count you can split by target.acceptedandreject_reasoncome from the content check, not the status code.
Ship the rows to whatever you already run for logs or metrics. The format matters less than having every attempt in one place with the same fields.
What Do You Do When Proxy Analytics Show Trouble?
A worked example of why the content check matters. A price-monitoring job runs overnight against one retailer, and it is the only traffic on its Node4 account during those hours, so the account's hourly figures are this job's figures. In the morning, the share of 200 responses for that target has gone up, latency is flat, and Node4's bandwidth for the run is about the same as the night before. Read from status codes alone, the run looks healthier than usual. But the accepted-results count for that target has fallen by half, and the reject_reason column is mostly challenge_page.
The target is answering, with a page your job cannot use. Work through it in order:
- Confirm the scope. Filter accepted results by target. If only one target fell, the proxy network as a whole is unlikely to be the cause; if every target fell at once, look at your client and the proxy path first.
- Read the bodies. Open a sample of rejected responses. A challenge page, a login wall, a consent page and a "no results" page each point to a different cause, and only the body tells you which.
- Check what changed on your side. Request rate, concurrency, headers, cookies or session state, and any account the job logs into. A change in your own deploy window is a more common cause than a change in the network.
- Check the target. A redesign or a new bot-management rule can change what the same request receives.
- Check the cost. With bandwidth flat and accepted results halved, bytes per accepted result has roughly doubled, because the job is paying for bandwidth that returned nothing usable. That is the number to put in front of whoever owns the budget. It holds here because the job ran alone; on an account shared with other work, see the attribution notes in the next sections.
Only after those checks is it worth changing how the job presents itself, its request rate or its proxy type. The symptom alone does not say which fix is right, and a status-code dashboard would have reported the night as a success.
How Do You Set Alert Thresholds That Don't Cry Wolf?
A fixed threshold such as "alert above 2 seconds" fires constantly where normal load varies by hour and target. Compare against your own recent baseline instead, chosen for the workload:
- Pick a window that covers the job's normal cycle. A job that runs nightly needs several nights of history; one that runs continuously may need a week to include weekday and weekend patterns. There is no universal number of hours; re-check the baseline after any change to the workload, the target or the proxy product.
- Require enough samples. A p99 from thirty requests is noise. Alert on percentiles only for targets with enough attempts in the window, and on counts for the rest.
- Count missing responses. Timeouts and connect failures never enter a response-latency percentile, so the percentile can improve while failures climb. Alert on failure share and time to failure alongside latency.
- Alert on sustained deviation. Require the change to persist across more than one evaluation window before paging anyone.
- Look for corroboration. Rising latency with a falling accepted-results count is a problem. Rising latency alone may just be a slow target.
- Watch retry load. Retries can hold the accepted count steady while traffic and cost climb. Chart retries per accepted result next to the success rate.
Does Node4 Cover the Full Proxy Analytics Checklist?
Partly. Node4 covers the volume half, at account, proxy and login level, not per target. The dashboard's Analytics page shows bandwidth and request history, including per proxy, and consumption against your allowance is on the Dashboard. On plans with a bandwidth allowance, the Usage Alerts tab in Account Settings can send an email or call a webhook at 80% and at 100% of it. It is included with every plan rather than sold as an add-on. Analytics is one part of a larger console; the proxy dashboard management checklist covers the rest, from session control to billing.
The same data is in the REST API under the read:usage scope:
| Endpoint | Returns |
|---|---|
GET /usage | This month's and all-time upload, download and request totals, plus recent usage log entries, each with a proxyId and a reporting period |
GET /usage/hourly?hours=48 | Hourly entries for the whole account: hour, bandwidthBytes, requests (up to 168 hours) |
GET /usage/daily?days=30 | The same per UTC day (up to 90 days) |
GET /usage/by-proxy | Lifetime uploadBytes, downloadBytes, requests per proxy, by host and port; traffic that cannot be tied to one proxy is an account-level row |
GET /usage/analytics?months=6 | Per-month totals, oldest first |
GET /sub-users/{id}/usage | One login's bandwidth for the current UTC month; no hourly or daily series |
An hour or day that carried no traffic is absent from the hourly and daily series, so treat a missing entry as zero. Sum the hourly entries that share an hour value rather than assuming one entry per hour:
curl -s -H "Authorization: Bearer $NODE4_API_KEY" \
"https://node4.io/api/v1/usage/hourly?hours=48"To join the two halves, the numerator and the denominator must cover the same traffic: the same account or login, the same proxies, the same targets and the same hours. The hourly series is account-wide, so dividing it by one target's accepted results only works when that target's job is the only traffic in those hours. A worked version, with illustrative numbers: the price-check job runs alone from 01:00 to 04:00 UTC. Once those hours have closed and stopped changing, the hourly entries for 01:00, 02:00 and 03:00 sum to 1.8 GB, and the client log shows 9,000 accepted results with a timestamp in the same three hours. Bytes per accepted result is about 200 KB. Count accepted results by the hour the attempt ran, and use whole UTC hours, because that is how the series is bucketed.
When several jobs share the account, isolate them before dividing. Give the job its own sub-user login and read that login's monthly figure before the run and again a few minutes after it ends (usage reaches the counter in short batches), or run it on proxies no other job uses and read their usage. Otherwise, split by target with your client's bytes field and treat the result as an estimate of each target's share, not as Node4's billed figure. Getting started with the Node4 API covers creating a scoped key, and the API reference lists every usage field.
The other half is yours to instrument. A target's status codes, challenge pages and per-request latency travel inside an HTTPS tunnel that the proxy relays without reading, so they never reach Node4's reporting. Requests refused at the proxy with a 407 never reach a target, so count authentication failures in your client as well.
How Should You Handle Data Privacy in Proxy Analytics?
Your client logs can say a lot about what you collect and from where, so give them the access rules you would give any operational data. Most people need the aggregated success and latency views, not the raw rows. Keep credentials out of the rows entirely, log host names rather than full URLs unless a path is genuinely needed, and decide on a retention period from your own investigation, privacy and legal needs rather than a default.
Node4 keeps its own connection records on its side of the boundary. What they contain and how long they are kept is set out in the privacy policy.
Ready to Put This Scorecard to Work?
Start with one job against one target, run while nothing else uses the account. Add the attempt log above to its client, pull GET /usage/hourly for the same whole hours, and compute bytes per accepted result for that target. Add targets once the numbers make sense, using a separate login or separate proxies for each one you want to price. If the job runs on Node4's dedicated datacenter proxies, per-proxy usage can be matched to the proxy label in your rows. On the residential gateway and the rotating gateways, the exit changes underneath one login, so use the login, not the exit, as the label.