Docker Proxy Rotation: Secrets, Workers and Exit Checks

Set up Docker proxy rotation with a runnable Python worker, Compose secrets, exit-IP checks, and fixes for authentication and routing errors.

Instead of manually replacing proxy addresses between container runs, set up Docker proxy rotation inside your request worker and inject credentials through a mounted secret. This workflow gives you a runnable Python container, explicit proxy routing, and checks that distinguish changing endpoints from changing exit IPs.

TL;DR

  • Docker proxy rotation belongs in the request worker or upstream gateway, not in container restarts.
  • Use Docker Compose secrets to keep proxy credentials out of the image and Compose configuration.
  • Selecting another proxy endpoint does not guarantee a different exit IP.
  • Verify routing against a controlled endpoint before sending production traffic.

What Docker does and does not control

Docker packages your application. It does not automatically rotate the public address websites see. Restarting a container changes its process state, not necessarily its outbound internet identity.

A working setup separates three responsibilities: Docker runs the worker, the application selects a proxy endpoint, and the proxy service determines the exit. The rotating proxies for web scraping guide explains that last layer.

Node4's rotating gateway fits developers who want provider-managed exit selection instead of maintaining their own endpoint list. The trade-off is that your application must follow the gateway's rotation and session behavior rather than assume every request receives a unique address.

Define success before writing configuration: authenticated requests reach the intended destination through an approved proxy, without silently falling back to a direct connection.

Before you start

  • Install Docker Engine and the Docker Compose plugin. Have permission to build images, run containers, and reach your proxy host from the Docker host.
  • Obtain authenticated proxy URLs and an HTTPS verification endpoint you control or are authorized to use. The example supports HTTP proxy endpoints carrying HTTPS destination requests through CONNECT.
  • Check the rotation contract first. A rotating gateway can keep the same exit across requests. A new container, TCP connection, or endpoint selection is not proof of a new exit.

Use a dedicated working directory. Keep its credential file outside source control, shared build artifacts, and support attachments. These are baseline controls for the setup below, not a replacement for your organization's secret-management policy.

The example uses Python's standard library. It makes sequential requests, performs no automatic retries, and stops when a request fails. Those limits make routing failures easier to identify before you add concurrency.

Proxy selection model

Choose one model before configuring the worker. Both use the same container, but they put exit selection in different places.

ModelApplication configurationBest forAdvantageLimitation
Endpoint listMultiple authenticated proxy URLsExplicit endpoint selectionThe worker records which list entry it selectedDifferent entries can still share an exit
Rotating gatewayOne authenticated proxy URLProvider-managed exit selectionNo local endpoint inventory to maintainExit changes follow the gateway's rules

Use an endpoint list when endpoint selection must be visible in application logs. Use a rotating gateway when the upstream service should manage that selection. An endpoint list and a rotating gateway both leave exit assignment to the provider, and neither one removes a destination's own access policy or request limits.

Configure the credential file

  1. Create a directory and move into it:
mkdir docker-proxy-worker
cd docker-proxy-worker
umask 077
  1. Read an authenticated proxy URL without placing its value directly in shell history. These commands assume Bash:
read -rsp 'Authenticated proxy URL: ' PROXY_URL
printf '\n'
printf '%s\n' "$PROXY_URL" > proxies.txt
unset PROXY_URL
  1. Repeat the prompt to append another endpoint when using a list:
read -rsp 'Another proxy URL: ' PROXY_URL
printf '\n'
printf '%s\n' "$PROXY_URL" >> proxies.txt
unset PROXY_URL

Skip that third step for a single rotating gateway. Enter the full URL supplied for your proxy, including its scheme, authentication, host, and port. Percent-encode reserved characters inside username and password components when constructing a URL yourself.

  1. Create .dockerignore and .gitignore:
printf '%s\n' 'proxies.txt' '.git' > .dockerignore
printf '%s\n' 'proxies.txt' > .gitignore

Expected result: proxies.txt contains one endpoint per line, and Docker excludes it from the build context. Docker Compose will mount it separately at runtime.

A Compose secret sourced from a local file is still a local file. Anyone with sufficient host access can read it. Do not treat this configuration as an encrypted secret vault.

Configure the request worker

  1. Save this program as worker.py:
import json
import os
import time
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path


class RefuseRedirects(urllib.request.HTTPRedirectHandler):
    # A redirect could leave the proxy route or carry the proxy
    # credentials to another host, so stop instead of following it.
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None


def main():
    target = os.environ["TARGET_URL"]
    count = int(os.environ.get("REQUEST_COUNT", "5"))
    interval = float(os.environ.get("INTERVAL_SECONDS", "2"))
    timeout = float(os.environ.get("TIMEOUT_SECONDS", "20"))
    show_body = os.environ.get("SHOW_BODY") == "1"

    if urllib.parse.urlsplit(target).scheme != "https":
        raise ValueError("TARGET_URL must use HTTPS")
    if count < 1 or interval < 0 or timeout <= 0:
        raise ValueError("Invalid request count, interval, or timeout")

    proxies = [
        line.strip()
        for line in Path("/run/secrets/proxy_urls").read_text().splitlines()
        if line.strip()
    ]
    if not proxies:
        raise ValueError("Proxy list is empty")

    for proxy in proxies:
        parsed = urllib.parse.urlsplit(proxy)
        if parsed.scheme != "http" or not parsed.hostname or not parsed.port:
            raise ValueError("Each proxy must be an HTTP URL with a port")

    for index in range(count):
        slot = index % len(proxies)
        proxy = proxies[slot]
        opener = urllib.request.build_opener(
            urllib.request.ProxyHandler({"https": proxy}),
            RefuseRedirects(),
        )
        request = urllib.request.Request(
            target,
            headers={"User-Agent": "container-proxy-check"},
        )
        started = time.monotonic()
        try:
            with opener.open(request, timeout=timeout) as response:
                body = response.read(256) if show_body else b""
                record = {
                    "request": index + 1,
                    "proxy_slot": slot + 1,
                    "status": response.status,
                    "elapsed_seconds": round(time.monotonic() - started, 3),
                }
                if show_body:
                    record["verification_sample"] = body.decode(
                        "utf-8", errors="replace"
                    )
                print(json.dumps(record), flush=True)
        except urllib.error.HTTPError as error:
            print(json.dumps({
                "request": index + 1,
                "proxy_slot": slot + 1,
                "http_error": error.code,
            }), flush=True)
            raise SystemExit(1)
        except (urllib.error.URLError, TimeoutError, OSError):
            print(json.dumps({
                "request": index + 1,
                "proxy_slot": slot + 1,
                "error": "connection_failed",
            }), flush=True)
            raise SystemExit(1)

        if index + 1 < count:
            time.sleep(interval)


if __name__ == "__main__":
    main()
  1. Keep the defaults for your first run: 5 requests, 2 seconds between completed requests, and a 20-second socket timeout. These are example settings, not throughput recommendations. The socket timeout is not a strict deadline for the entire job.
  1. Review what the worker records. proxy_slot identifies a credential-file entry, not an exit IP. Logs omit proxy URLs and exception details to reduce accidental credential exposure.

Expected result: each request selects a configured endpoint. If that endpoint fails, the worker exits instead of deliberately retrying without a proxy.

The worker also refuses redirects. Python follows them by default, and a redirect to a plain HTTP address would leave the proxied HTTPS route and could send the proxy credentials straight to the destination. A 3xx response is reported as an http_error with its status code and stops the run. Point TARGET_URL at the final HTTPS address instead of following the redirect.

One inherited setting needs attention: Python's proxy handling can honor no_proxy or NO_PROXY. The Compose configuration below clears both so that an inherited bypass list does not defeat the test.

Configure the container and verification run

  1. Save this Dockerfile:
FROM python:3.13-slim
WORKDIR /app
COPY worker.py /app/worker.py
ENV PYTHONUNBUFFERED=1
CMD ["python", "-B", "/app/worker.py"]

The image contains code, not credentials. The base-image tag is mutable. Record the tested image digest and use an approved digest for repeatable deployments.

  1. Save this configuration as compose.yaml:
services:
  worker:
    build: .
    environment:
      TARGET_URL: ${TARGET_URL:?Set TARGET_URL first}
      REQUEST_COUNT: "5"
      INTERVAL_SECONDS: "2"
      TIMEOUT_SECONDS: "20"
      NO_PROXY: ""
      no_proxy: ""
    secrets:
      - proxy_urls
    read_only: true
    cap_drop:
      - ALL
    security_opt:
      - no-new-privileges:true

secrets:
  proxy_urls:
    file: ./proxies.txt
  1. Enter an HTTPS endpoint that reports the source IP it observes. Use a controlled endpoint whose response contains no sensitive information:
read -rp 'HTTPS verification URL: ' TARGET_URL
export TARGET_URL
docker compose build
docker compose run --rm -e SHOW_BODY=1 worker
  1. Compare the returned source addresses with a direct request made separately from the same Docker host. Use your endpoint's response format to identify the observed address:
curl --noproxy '*' --silent --show-error "$TARGET_URL"

Expected result: the container produces five JSON records, with a verification sample for each successful request. For a conventional remote proxy, the endpoint should observe the proxy exit rather than the host's direct address.

Do not interpret repeated addresses as an automatic failure. A gateway can reuse exits, and separate credentials can map to the same exit. Confirm the provider's rotation behavior before changing working code.

For the acceptance record, retain the tested image digest, selected endpoint slots, observed exits, and failure outcomes. Exclude credentials. A successful response proves connectivity; the observed source address proves the route.

Keep one endpoint for a multi-request job

Best for: workflows that need continuity across related requests. The default worker advances through the endpoint list on every request. For a job that must keep one endpoint, select the list entry once before the loop.

Add this immediately before for index in range(count)::

    job_slot = int(os.environ.get("JOB_SLOT", "0")) % len(proxies)

Then replace the slot-selection line inside the loop:

        slot = job_slot

Run a job against the first entry with:

docker compose run --rm -e JOB_SLOT=0 worker

This pins the configured endpoint, not necessarily its exit IP. A provider-managed gateway still needs its documented sticky-session configuration. Do not invent session parameters or append guessed username suffixes.

This variant also does not preserve cookies: the sample worker has no cookie jar. Add explicit cookie handling when the destination requires it, and bind that state to the same logical job. Read the Python requests session workflow before extending the worker into a stateful collector.

Troubleshooting

Proxy authentication fails

An HTTP 407 response indicates that proxy authentication is required or was not accepted. Recheck the username, password, endpoint, and port against your supplied connection details. Reserved characters in URL credentials need correct encoding.

Do not paste the complete proxy URL into an issue tracker. Share the status and selected slot instead.

The destination sees the host's direct IP

Check NO_PROXY and no_proxy inside the container. Confirm that the request uses the configured opener rather than a separate library call. A host-level proxy test does not prove that container traffic follows the same route.

Keep both bypass variables empty for this verification workflow.

Requests fail with a connection error

Check DNS resolution, outbound firewall rules, the proxy port, and any restricted source-IP policy on the proxy host. Container egress can differ from the address you used when configuring access.

The sample deliberately groups connection failures into one safe log message. Diagnose the specific failure in a restricted environment without exposing credentials in shared logs.

The exit address never changes

Check whether you configured static endpoints, a rotating gateway, or sticky credentials. Restarting Docker is not a rotation instruction. Test across the provider's documented rotation boundary rather than assuming a fresh connection creates a fresh exit.

The destination returns 403 or 429

A 403 response is a refusal; 429 indicates too many requests. Neither proves that the proxy transport failed. Inspect the destination's response and access policy, reduce request pressure, and respect Retry-After when supplied.

Do not add unlimited retries. They multiply traffic without resolving the underlying restriction.

Customize your workflow

Before raising concurrency in your deployment, separate transport health from collection success. Record endpoint slot, destination status, elapsed time, and a job identifier. Track verification exits separately from production response content.

The sample reads only a small verification response when requested. A production collector must consume or stream the destination body deliberately, validate its contents, and store results. HTTP success alone does not establish that the expected data arrived.

Next, add a bounded worker queue, explicit retry limits, and destination-specific pacing. Run the container as a non-root user after arranging readable secret-file permissions for that user. Test that change rather than assuming a host file remains readable after switching identities.

Keep rotation policy separate from retry policy. A scheduled endpoint change and a recovery attempt are different events. Logging them separately makes failures easier to explain.

FAQ

How do I set up Docker proxy rotation?

Select a proxy endpoint in the request worker or use an upstream rotating gateway. Mount credentials at runtime, route requests explicitly through the proxy, and verify the source address at a controlled destination.

Does restarting a Docker container change its public IP?

Restarting a Docker container does not guarantee a different public IP. The observed address depends on the host's outbound network path and any proxy used by the application.

Should I use a rotating gateway or a list of proxies?

Use a rotating gateway for provider-managed exit selection and an endpoint list for explicit application-level selection. Both options leave the exit assigned to each request up to the gateway's or the list's own rules, not to a fixed one-to-one mapping.

Can I use SOCKS5 with this Python example?

No, this example accepts HTTP proxy URLs only. SOCKS5 requires a compatible client or additional transport support, plus a separate routing verification test.

Why does my container keep using the same exit IP?

Repeated exit IPs can result from static endpoints, sticky sessions, or gateway reuse. Check the upstream rotation contract before treating repeated addresses as a container failure.

Are Docker Compose secrets encrypted automatically?

A Compose secret sourced from a local file is not automatically an encrypted secret vault. Protect the source file and Docker host, and use your deployment platform's secret-management controls.

Can one proxy handle multiple containers?

Multiple containers can use the same proxy credentials when the provider's access rules permit it. Their combined connections and traffic still count against the applicable account or endpoint limits.

Test the negative case before scaling

Test failure before scaling success. In an isolated test, replace the proxy port with a known closed port and run one request. The worker should fail, and your controlled destination should receive no request from that run.

That negative test checks the property an IP echo alone cannot establish: your application does not silently switch to a direct route when its proxy becomes unavailable.