Deploys

Preview checks

Your monitors already say what “working” means in production. Preview checks ask the same questions of every pull request's preview deployment and put the answer where the agent and the reviewer will see it, before the merge.

#How it works

1. Connect a deploy hook. The same URL that reports production deploys also receives the provider's preview deployments (Vercel previews, Netlify deploy previews, Cloudflare Pages previews, Railway PR environments, Coolify pull request previews, GitHub deployment_status for any environment that is not production, or a generic body with "environment": "preview").
2. For a finished preview deploy, UpButler takes the HTTP monitors mapped to that project and points each one at the preview: the origin is swapped, the path, query, method, body and assertions stay. https://acme.com/checkout becomes https://acme-web-git-fix-acme.vercel.app/checkout.
3. Each check runs once, from the main region. A failing check is tried one more time after a few seconds, because previews answer their first request cold. Nothing touches the monitor itself: no state change, no incident, no alert, no check history.
4. Every result is compared with the monitor's latest production result, and the run gets a verdict: pass, fail or error.
5. The verdict is reported as a GitHub commit status, a signed preview.verdict event, a row on the Deploys page, and through previews_get for an agent that asks “is my PR green?”.

Preview checks are on by default as soon as a deploy hook is connected, on every plan (the Free plan includes 25 runs a month). There is nothing to configure for a Vercel, Netlify or Cloudflare Pages project whose monitors are on its production domain.

The hook answers a preview delivery at once; the checks run a moment later:

{
  "ok": true,
  "outcome": "preview",
  "provider": "vercel",
  "event": "deployment.succeeded",
  "previewRunId": "pvr_0n3qc1a2b3c4d5e6f7g",
  "previewUrl": "https://acme-web-git-fix-rounding-acme.vercel.app",
  "checks": 4,
  "verdictUrl": "https://upbutler.com/api/v1/previews/pvr_0n3qc1a2b3c4d5e6f7g"
}

#The verdict

Each check ends in one of five results. Only fail fails the verdict.

Verdict pass: no check failed. fail: at least one did. error: the preview could not be tested at all; this says nothing about the change, and verdict.summary names what to set up.

The diff against production on each check has status (production vs preview status code, when they differ), latencyMs (both values and the delta), outcome, and failedAssertions (the assertions that fail on the preview). “Production” is the monitor's latest scheduled check, so no extra request goes to your live site. TLS-expiry and “slower than N ms = degraded” settings are left out on previews.

#For agents: is my PR green?

After gh pr create or a push to a branch, poll previews_get (MCP) or GET /api/v1/previews/latest with pr, commit (a prefix is enough) or branch until state is no longer pending.

curl "https://upbutler.com/api/v1/previews/latest?pr=212" \
  -H "Authorization: Bearer $UPBUTLER_API_KEY"

{
  "found": true,
  "state": "fail",                       // pending | pass | fail | error | none
  "summary": "1 of 4 checks fails on the preview of PR #212 (abc1234): Checkout: Expected status 200-299, got 500 Internal Server Error (production: 200).",
  "next": "Do not merge. Fix what the failing checks name, push, and ask again for the new commit.",
  "run": {
    "_id": "pvr_0n3qc1a2b3c4d5e6f7g",
    "state": "fail",
    "previewUrl": "https://acme-web-git-fix-rounding-acme.vercel.app",
    "provider": "vercel", "project": "acme-web",
    "commit": "abc1234def5678901234567890abcdef12345678", "branch": "fix/rounding", "pr": 212, "repo": "acme/acme-web",
    "verdict": { "status": "fail", "total": 4, "passed": 3, "failed": 1, "warned": 0, "preexisting": 0, "blocked": 0 },
    "checks": [
      {
        "monitorId": "mon_0n3qc1a2b3c4d5e6f7g",
        "name": "Checkout",
        "method": "GET",
        "url": "https://acme-web-git-fix-rounding-acme.vercel.app/checkout",
        "result": "fail",                // pass | fail | warn | preexisting | blocked
        "preview":    { "outcome": "down", "statusCode": 500, "latencyMs": 412, "error": "Expected status 200-299, got 500 Internal Server Error", "assertions": [] },
        "production": { "state": "up", "outcome": "up", "statusCode": 200, "latencyMs": 131, "at": "2026-10-10T09:11:02.000Z" },
        "diff": { "changed": true, "outcome": { "production": "up", "preview": "down" }, "status": { "production": 200, "preview": 500 },
                  "latencyMs": { "production": 131, "preview": 412, "delta": 281 }, "failedAssertions": [] },
        "headersSent": false, "attempts": 2
      }
    ],
    "dashboardUrl": "https://upbutler.com/app/deploys/previews/pvr_0n3qc1a2b3c4d5e6f7g"
  }
}

The Claude Code plugin tells the agent this by itself: after a push to a non-default branch or gh pr create it adds one note naming the exact previews_get call, when a deploy hook is connected and preview checks are on. previews_get is in the full MCP toolset; it is not part of ?toolset=core, but a core session can still call it by name.

#Run it yourself

No deploy hook, or a provider that reports nothing? Check a preview URL on request, from CI or an agent. The call waits for the verdict by default, so a pipeline can gate on it:

curl -fsS -X POST https://upbutler.com/api/v1/previews \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"url\": \"$PREVIEW_URL\", \"project\": \"acme-web\", \"commit\": \"$GITHUB_SHA\", \"pr\": $PR_NUMBER}" \
  | jq -e '.state == "pass"'

#GitHub commit status

Give UpButler a token and every run sets a commit status on the pull request's head commit: pending when the checks start, then success, failure or error, with a one-line description and a link to the run. Require the status in branch protection and a broken preview cannot be merged.

1. GitHub → Settings → Developer settings → Fine-grained tokens → Generate new token.
2. Repository access: only the repositories you deploy. Permissions → Repository → Commit statuses: Read and write. Nothing else.
3. Paste it under Settings → Deploy hooks → Preview checks, or send it as github.token to PATCH /previews/settings.

The token is stored encrypted and never returned. The status is named UpButler / preview (change it with github.context). It is sent with POST /repos/{owner}/{repo}/statuses/{sha}, so the run needs the repository (from the provider payload, a "repo" field, or the repo: of your upbutler.yaml) and the full 40-character commit SHA. If GitHub refuses, the settings page and the run say why in words.

#Check Runs with the GitHub App

With the UpButler GitHub App installed on the repository, the verdict is a Check Run instead: no token to paste, and its summary is a table of every check with the preview result next to production. fail concludes as failure, a preview that could not be tested as neutral. The App also sets the commit status UpButler / preview status, so a branch protection rule can require it (or the Check Run UpButler / preview) without a token. If a token is saved, the status is set with that token and its name instead.

#A failing preview goes back to the agent

When the verdict is fail on a pull request that Fix with Claude opened, the agent is paged with the failing checks and their production diff and pushes a fix commit to that pull request's branch. For every pull request: turn on Fix failing previews, or ask for one run with POST /previews/:id/fix.

#The preview.verdict event

Every finished run emits preview.verdict. It is in the event stream and is delivered, signed, to every alert channel that lists it (or *). Channels do not get it by default: twenty pull requests a day should not land in the on-call channel unasked. Slack, Discord, Telegram and email channels get a readable message with the failing checks.

{
  "id": "evt_0n3qc1a2b3c4d5e6f7g",
  "type": "preview.verdict",
  "timestamp": "2026-10-10T09:12:44.000Z",
  "workspaceId": "ws_…",
  "data": {
    "preview": {
      "id": "pvr_0n3qc1a2b3c4d5e6f7g",
      "state": "fail",
      "verdict": { "status": "fail", "summary": "1 of 4 checks fails on the preview of PR #212 (abc1234): …", "total": 4, "passed": 3, "failed": 1, "warned": 0, "preexisting": 0, "blocked": 0 },
      "previewUrl": "https://acme-web-git-fix-rounding-acme.vercel.app",
      "provider": "vercel", "project": "acme-web", "environment": "preview",
      "commit": "abc1234def5678901234567890abcdef12345678", "branch": "fix/rounding", "pr": 212, "repo": "acme/acme-web",
      "checks": [ { "monitorId": "mon_…", "name": "Checkout", "result": "fail", "url": "…/checkout",
                    "preview": { "outcome": "down", "statusCode": 500, "latencyMs": 412, "error": "Expected status 200-299, got 500 Internal Server Error" },
                    "production": { "outcome": "up", "statusCode": 200, "latencyMs": 131 },
                    "diff": { "changed": true, "status": { "production": 200, "preview": 500 }, "failedAssertions": [] } } ],
      "url": "https://upbutler.com/api/v1/previews/pvr_0n3qc1a2b3c4d5e6f7g",
      "dashboardUrl": "https://upbutler.com/app/deploys/previews/pvr_0n3qc1a2b3c4d5e6f7g"
    }
  }
}

#Which monitors run

• Only HTTP monitors (status, keyword, JSON, header and latency assertions). TCP, DNS, heartbeat, manifest, script, browser and AI monitors are skipped: they have no origin to swap, or carry their URLs inside code.
• By default, the monitors the deploy mapping gives for the project, narrowed to the ones on the project's production domain. A monitor of another origin in the same mapping (a payment provider, the docs site) is left out, because it could only fail against this preview.
• Opt a monitor out with exclude in the settings or the file, or by giving it the tag no-preview.
• List monitors explicitly with monitorIds (previews.monitors in the file) when the automatic choice is not what you want.
• Paused monitors never run. A run checks at most 40 monitors.

#Security: what a preview may receive

• Request headers stay home. Monitors can store headers (API keys, cookies, basic auth). They are not sent to previews unless you turn on sendHeaders. When a check fails with 401 or 403 for that reason, the run says so. Turn it on only when your previews are private and reviewed code is all that deploys there.
• The preview host must be expected. A URL from a webhook is accepted only when it is on the provider's preview domain and carries the project's name (acme-web-….vercel.app, …--acme-docs.netlify.app, ….acme-landing.pages.dev, api-….up.railway.app; for Coolify, a subdomain of the application's own monitored domain such as 212.acme.com), or when it matches your allowedHosts, or when it comes out of your own urlTemplate. Anything else is refused and the delivery log says why. A wildcard on a shared hosting domain must pin your own name: acme-*.vercel.app is accepted, *.vercel.app is not.
• Production is never a preview. A “preview” whose host is one of your monitored production hosts is refused.
• No private addresses. Plain http://, IP literals and hosts that resolve to private or link-local ranges are refused, exactly as for monitors (self-hosted installs opt in with ALLOW_PRIVATE_TARGETS).
• Stored URLs lose their query string, and the response body is not kept.

#Vercel Deployment Protection

Vercel protects preview deployments by default: a request without a login gets a 401 page. UpButler recognises that page, marks the checks blocked and the verdict error instead of failing your pull request. To let the checks through:

1. Vercel → Project → Settings → Deployment Protection → Protection Bypass for Automation → add a secret.
2. Paste it under Preview checks → Vercel bypass secret (or vercelBypassSecret in PATCH /previews/settings).

It is stored encrypted, never returned, and sent as the x-vercel-protection-bypass header only to previews reported by Vercel or served from *.vercel.app.

#Where the preview URL comes from

A URL template covers the rest and wins over the payload: https://pr-{pr}.preview.acme.dev. Placeholders: {pr}, {branch}, {commit}, {commit7}, {project}, {deploymentId}, {deploymentId8}. Values are reduced to letters, digits and dashes, and the domain itself must be literal, so a branch name cannot point the checks somewhere else.

curl -fsS -X POST "https://upbutler.com/hooks/deploy/<token>" \
  -H "Content-Type: application/json" \
  -d "{\"service\": \"api\", \"environment\": \"preview\", \"previewUrl\": \"https://pr-$PR.preview.acme.dev\",
       \"commit\": \"$GIT_SHA\", \"branch\": \"$BRANCH\", \"pr\": $PR, \"repo\": \"acme/api\"}"

#Settings

In the dashboard: Settings → Deploy hooks → Preview checks. Through the API (only the fields you send change):

curl -X PATCH https://upbutler.com/api/v1/previews/settings \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "allowedHosts": ["pr-*.preview.acme.dev"],
    "exclude": ["mon_0n3qc1a2b3c4d5e6f7g"],
    "github": { "token": "github_pat_…" }
  }'

In upbutler.yaml:

# upbutler.yaml
version: 1
project: acme-web
monitors:
  - id: homepage
    url: https://acme.com/
  - id: checkout
    url: https://acme.com/checkout
    keyword: "Pay now"
  - id: admin
    url: https://acme.com/admin
previews:
  enabled: true            # default
  exclude: [admin]         # login-only: makes no sense on a preview
  sendHeaders: false       # default: stored request headers stay home
  # monitors: [homepage, checkout]            # only these (default: automatic)
  # allowedHosts: ["pr-*.preview.acme.dev"]   # hosts beyond the provider's own pattern
  # urlTemplate: https://pr-{pr}.preview.acme.dev

The file's monitors and exclude lists replace only the entries of that project; the token and the bypass secret are workspace settings and are never written to the file.

#Retries

One deployment is one run. Providers retry webhooks and send several events per deployment; they all find the same run (keyed on the provider's deployment id, an Idempotency-Key header, or project + commit + URL), spend one run of the allowance and produce one verdict. A new commit on the same pull request is a new deployment and a new run.

#Plans

A run is one preview deployment, however many monitors it checks. When the month's runs are used up, further previews are logged as ignored with that reason and no status is posted; nothing else stops working. GET /previews/settings returns usage.

#API

Runs are kept for 90 days.