Monitoring

Private monitoring

A private probe is a small container in your network. It asks UpButler for work over HTTPS, runs the checks where your internal systems live, and reports back. No inbound ports, no VPN, no agent with database access.

#How it works

1. You create a private probe (dashboard: Monitors → Private probes; API: POST /probes). You get a token, once.
2. You run the probe image with that token. It makes outbound HTTPS calls only to UpButler: claim work (a long poll), run the checks, report results.
3. Monitors pick the probe under Check from (or "regions": ["prod-vpc"]). A monitor that lists only private probes is checked from inside your network and may target private addresses and internal hostnames.

curl -X POST https://upbutler.com/api/v1/probes \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "prod-vpc"}'

{
  "id": "prv-7k2m9x4q1h8c",
  "name": "prod-vpc",
  "status": "never_connected",
  "token": "ubp_3fJ…",                    // shown once
  "run": "docker run -d --name upbutler-probe --restart unless-stopped -e UPBUTLER_URL=https://upbutler.com -e PROBE_TOKEN=ubp_3fJ… upbutler-probe"
}

#Run the probe

The probe is the same code as UpButler's public check regions, bundled into one file by Dockerfile.probe (an oven/bun image, no dependencies, no database access). There is no published image yet: build it from the repository.

# in a checkout of the UpButler repository
docker build -f Dockerfile.probe -t upbutler-probe .

docker run -d --name upbutler-probe --restart unless-stopped \
  -e UPBUTLER_URL=https://upbutler.com \
  -e PROBE_TOKEN=ubp_3fJ… \
  upbutler-probe

docker logs upbutler-probe
# [probe:prod-vpc] connected to https://upbutler.com as a private probe (server time …)

The probe needs to reach UPBUTLER_URL on 443 and whatever it is asked to check. It needs no inbound port, no Docker socket and no other secret. Several containers may share one token (for redundancy): leases keep them from running the same check twice. To run it without Docker: bun build src/server/probe/main.ts --target bun --outfile probe.js, then bun probe.js with the same environment.

#What a private probe can check

Kinds http, tcp, dns, manifest, mcp, llm and proxy. Script and browser checks need UpButler's sandbox and stay on the main region; heartbeats and other push-fed kinds have nothing to probe.

# An internal HTTP service
curl -X POST https://upbutler.com/api/v1/monitors -H "Authorization: Bearer $UPBUTLER_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "Billing API (internal)", "url": "http://billing.internal:8080/health", "regions": ["prod-vpc"]}'

# A database port
  -d '{"name": "Postgres primary", "host": "10.0.3.12", "port": 5432, "regions": ["prod-vpc"]}'

# An internal MCP server
  -d '{"name": "Ops MCP", "kind": "mcp", "url": "http://mcp.internal:3000/mcp", "regions": ["prod-vpc"]}'

# A forward proxy that only your network can reach
  -d '{"name": "Squid egress", "proxy": {"url": "http://user:[email protected]:3128"}, "regions": ["prod-vpc"]}'

• Internal services: any HTTP endpoint, with the same assertions as a public monitor.
• Databases and queues: a tcp monitor on the port (Postgres 5432, MySQL 3306, Redis 6379, Kafka 9092). It proves the port accepts connections, not that queries work; expose a health endpoint for that.
• Proxies: an internal forward proxy, or a commercial pool as seen from your own egress.
• Internal MCP servers and LLM gateways: handshake, tool list and schema drift; first-token time through your gateway.
• Internal DNS: a dns monitor uses the resolver of the machine the probe runs on.

#Trust model

• Token scope. A probe token can do three things: say it is alive, claim jobs of monitors in its own workspace that list this probe, and report results for the rounds it claimed. It is not an API key: it cannot read monitors, incidents or anything else.
• Tenancy. The workspace is part of every query the probe protocol runs. A probe never receives another workspace's job and cannot report into another workspace's check, whatever a monitor's configuration says; a monitor can only name probes of its own workspace.
• Private targets are allowed on your probe only. UpButler's own regions keep refusing private addresses, internal hostnames and redirects to them. Jobs for a private probe are marked as allowed to reach them; no other job is.
• The token is hashed at rest and shown once. Rotate replaces it at once (the running probe gets 401 until restarted with the new one). Delete removes the probe; it is refused while monitors still use it, so a private target is never handed to a public region by accident.
• Rate limits. 600 probe API calls a minute per probe; failed authentications are limited per address.
• What the probe sends back is treated as input: results are size-capped and only known evidence fields are kept.

#When a probe is offline: no data, not down

A probe that has not asked for work for 90 seconds takes no part in new checks. If no other region of a monitor can check it, the monitor shows No data · probe offline (dataState: "no_data", noData.since), keeps its last known state, opens no incident and does not change its status page components. After 3 minutes the team gets one alert; when the probe returns, checks resume at once and a second message says so.

📡 Private probe "prod-vpc" is offline
3 monitors have no data until it is back. Their targets are not reported as down: nothing is checking them.
Last seen   2026-10-10 14:02 UTC
Monitors    Billing API (internal), Postgres primary, Squid egress
Next        Check the container on your side: docker logs upbutler-probe

Events: probe.offline and probe.online (both default channel events). A probe that was created and never started sets "no data" without an alert. GET /probes lists status, last seen, version and checks run.

"Test now" on such a monitor asks its probe for a check and waits for the answer; with the probe offline it says so instead of testing from a public region.

#Confirmation and quorum

Private probes do not count toward the plan's regions per monitor. Uptime measured by a private probe is yours, not UpButler's: such monitors are listed as self-reported in verified uptime attestations.

#Keeping it off public pages

• A monitor is on a status page only when you attach it to a component. Internal monitors need no component: alerts, incidents (internal ones) and the dashboard work without.
• If you do want "Internal tools: operational" on a page, attach the monitor to a component with a name of your choosing. Visitors see the component's name and status; hostnames, addresses, check errors and probe names stay in the dashboard. Automatic incidents use the component's name.
• Mark a component hidden to track it without showing it, or keep internal components on a private page.
• Proxy monitors go further: their name, hosts and provider are never published, whatever they are attached to.

#In upbutler.yaml

Monitors refer to private probes by name with probes:. Probes themselves are created in the dashboard or with the API: a token cannot live in a file.

monitors:
  - id: billing-internal
    name: Billing API (internal)
    url: http://billing.internal:8080/health
    probes: [prod-vpc]            # private probes by name

  - id: postgres-primary
    host: 10.0.3.12
    port: 5432
    probes: [prod-vpc, dr-site]   # two probes: both must agree before it is down

  - id: public-api
    url: https://api.example.com/health
    regions: [eu-central, de-nbg] # public regions, as before
    probes: [prod-vpc]            # plus the view from inside

An unknown probe name fails the apply before anything is written. upbutler export writes probes by name, so the file moves between workspaces that have a probe of the same name.

#Plans

After a downgrade the oldest probes within the new plan keep working. A monitor left with no probe is paused, never moved to a public region.

#API

Creating, rotating and deleting a probe needs the owner or admin role in the dashboard, or an API key with the write scope.