Monitoring

Proxy monitors

A scraper is only as healthy as its egress. A proxy monitor sends real requests through your proxies and tells you which one is failing and why: the proxy, the provider account, or the target.

#How it works

1. For every proxy in the pool, each check sends samples requests to the target through that proxy: absolute-form or CONNECT for http:// and https:// proxies, the RFC 1928 handshake for socks5://. The proxy resolves the target's name, as it does for your own traffic.
2. Each request is timed (connect to the proxy, tunnel, first byte, total) and, when it fails, given a failure class.
3. The proxy is healthy when at least minSuccess requests succeeded and the exit expectations hold (country, distinct IPs, latency).
4. The monitor is down when fewer than minHealthyPct percent of the proxies are healthy, and degraded while any of them is failing. The usual confirmation applies (failureThreshold, default 2 checks).

curl -X POST https://upbutler.com/api/v1/monitors \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Evomi residential",
    "proxy": {
      "url": "http://'"$EVOMI_USER"':'"$EVOMI_PASS"'_country-US_session-{session}@rp.evomi.com:1000",
      "samples": 10, "minSuccess": 8, "minDistinctIps": 5, "expectCountry": "US"
    }
  }'

{
  "_id": "mon_0n3qc1a2b3c4d5e6f7g",
  "kind": "proxy",
  "target": "http proxy via rp.evomi.com",
  "proxy": {
    "entries": [
      { "id": "rp-evomi-com-1000", "name": "rp.evomi.com:1000", "scheme": "http", "host": "rp.evomi.com", "port": 1000,
        "hasCredentials": true, "credentialHint": "fet…", "rotatesSession": true, "provider": "Evomi" }
    ],
    "target": { "url": "https://upbutler.com/api/v1/echo/ip", "echo": true, "method": "GET", "expectedStatus": "200-299" },
    "samples": 10, "minSuccess": 8,
    "expect": { "country": "US", "minDistinctIps": 5 },
    "minHealthyPct": 50
  }
}

#The target

By default the target is UpButler's own IP echo, GET /api/v1/echo/ip, which answers {"ip": "…", "country": "US"}: the exit IP and country of the request, so rotation and country rules work without configuration. Set target to fetch something else: the site you scrape (to notice blocks), or your provider's echo (https://ip.evomi.com/s, https://geo.brdtest.com/mygeo.json, https://ip.oxylabs.io/location, https://ip.decodo.com/json, https://ipv4.webshare.io/). Exit IP and country are read from plain-text and JSON echo answers; a target that reports neither cannot be used with the country and rotation rules, and the proxy shows exit_ip_unknown.

#Pools

One monitor can hold several proxies. Each is a row on the monitor page with its own result, and the monitor carries the verdict for the pool.

{
  "name": "Egress pool",
  "proxy": {
    "proxies": [
      { "name": "evomi-residential", "url": "http://USER:PASS_session-{session}@rp.evomi.com:1000" },
      { "name": "evomi-datacenter",  "url": "http://USER:[email protected]:2000" },
      { "name": "evomi-socks",       "url": "socks5://USER:[email protected]:1002" }
    ],
    "samples": 5,
    "minHealthyPct": 67
  }
}

When the proxies are one endpoint with different parameters, declare a template: every variant becomes a pool entry, and a variant with vars.country expects that exit country.

{
  "name": "Evomi residential by country",
  "proxy": {
    "template": {
      "url": "http://USER:PASS_country-{country}_session-{session}@rp.evomi.com:1000",
      "variants": [
        { "vars": { "country": "US" } },
        { "vars": { "country": "DE" } },
        { "name": "uk", "vars": { "country": "GB" } }
      ]
    },
    "samples": 6, "minSuccess": 5, "minDistinctIps": 3
  }
}

{session} is replaced by a fresh random 8-character value on every request, so a provider that keys sticky sessions on it hands out a new exit each time. Without it, set a rotating endpoint or expect one IP.

On update, proxies replaces the list. An entry sent with its id (or name) and no url keeps the stored URL; an entry left out is removed. GET /monitors/:id/proxy returns the pool with the latest result per proxy and region:

{
  "target": { "url": "https://upbutler.com/api/v1/echo/ip", "echo": true, "method": "GET", "expectedStatus": "200-299" },
  "samples": 10, "minSuccess": 8, "expect": { "minDistinctIps": 5 }, "minHealthyPct": 50,
  "entries": [{ "id": "us", "name": "us", "scheme": "http", "host": "rp.evomi.com", "port": 1000, "hasCredentials": true, "credentialHint": "fet…", "provider": "Evomi" }],
  "regions": [
    { "region": "eu-central", "label": "Helsinki", "at": "2026-10-10T10:05:00.000Z", "outcome": "down", "healthy": 0, "total": 1,
      "entries": [{ "id": "us", "name": "us", "proxy": "http://rp.evomi.com:1000", "outcome": "down",
        "failure": "proxy_auth_failed", "message": "Proxy refused the tunnel: 407 Proxy Authentication Required",
        "ok": 0, "samples": 10, "failures": { "proxy_auth_failed": 10 }, "exitIps": [], "connectMs": 41, "totalMs": 92 }] }
  ]
}

#Failure classes

The classes appear per proxy on the monitor page, in the alert, in the AI incident report and in the packet an agent responder receives (hosts and timings, credentials scrubbed).

#Provider-specific answers

Providers refuse requests with their own codes. Where their documentation states the shape, UpButler recognises it and names the provider in the message; everything else falls back to the generic rules (407 = authentication, 402 = out of credits, 429 = rate limited, 502 / 503 / 504 = target unreachable).

Bright Data is matched on its x-brd-err-code header, Oxylabs on X-Error-Description, Decodo on x-error-message; IPRoyal and Evomi on the few statuses their docs define, and only on their own hosts. SOCKS5 reply codes are not documented by any of them and are read per RFC 1928.

#Never public

• A proxy monitor is on no status page unless you attach it to a component. Then visitors see that component's name and status and nothing else: no monitor name, no proxy host, no provider, no error text.
• That holds for every public surface: the page, its JSON, status.agent.json, the page MCP server, llms.txt, RSS and Atom feeds, subscriber notifications.
• An automatic incident for a proxy monitor is titled by its components, and AI-written public text that names the monitor, a pool entry, a proxy host or the provider is not published (the template text is used instead).

#Regions, private probes and what is refused

Proxy monitors run from the main region, from other regions (a proxy that works from Helsinki and not from Ashburn is a finding), and from private probes inside your network.

From public regions, UpButler connects only to proxies on public addresses and only fetches public hostnames through them: a proxy is an egress, and it must not be a way to reach an internal address. A private IP literal, localhost, a single-label or .internal / .local name, and a public-looking name that resolves to a private address are refused (refused), at save time and again on every check. To monitor an internal proxy or reach an internal target through one, run the monitor on a private probe only.

#In upbutler.yaml

monitors:
  - id: evomi-residential
    name: Evomi residential
    kind: proxy
    intervalSec: 300
    proxy:
      proxies:
        - name: us
          url: ${env:EVOMI_PROXY_US}      # http://user:pass_country-US_session-{session}@rp.evomi.com:1000
          expectCountry: US
        - name: de
          url: ${env:EVOMI_PROXY_DE}
          expectCountry: DE
      samples: 10
      minSuccess: 8
      minDistinctIps: 5
      maxLatencyMs: 4000

  # The same pool, checked from inside your network by a private probe:
  - id: squid-internal
    kind: proxy
    probes: [prod-vpc]
    proxy:
      url: ${env:INTERNAL_SQUID_URL}      # http://user:[email protected]:3128
      target: http://billing.internal/health

${env:NAME} is read from your environment by upbutler apply, never from a file. upbutler export writes each proxy by id without its URL; applying such a file keeps the stored URLs.

#Limits and cost

• A proxy monitor counts as one monitor. Requests per check: samples × proxies × regions, at most 20 samples per proxy, ten requests in flight at a time. Every request uses your proxy traffic.
• The check timeout (default 30 seconds) covers the whole check. Requests that could not start in time count as timeout: lower samples or raise the timeout for large pools.
• Responses are read up to 256 KB; redirects are not followed (set expectedStatus accordingly).
• The echo endpoint is rate-limited to 120 requests a minute per exit address.

#Fields

The proxy object of POST /monitors and PATCH /monitors/:id: