Incidents & alerts

Teams, PagerDuty, Opsgenie & push

Four alert channel types for teams that already live in Microsoft Teams or a paging tool, and for people who want the alert on their phone without installing another app.

All four are ordinary alert channels: create them under Alerts in the dashboard or with POST /channels, attach them to monitors, filter events, and try them with Send test. Every alert leads with the same plain sentence and carries the same signed, confirm-first links: Roll back, Let Claude fix, Acknowledge, Stop run and Mute 1h.

#Microsoft Teams

Microsoft retired Office 365 connectors (the old https://<tenant>.webhook.office.com/… incoming webhooks). The supported way to post into a channel from outside is a Workflows flow (Power Automate) with the trigger When a Teams webhook request is received. UpButler rejects a connector URL with a message that says so.

1. In Teams, go to the team and channel that should get the alerts.
2. Select More options (…) next to the channel name, then Workflows.
3. Search for the template Send webhook alerts to a channel and select it. (Older tenants list it as Post to a channel when a webhook request is received.)
4. Give the flow a name, check the team and channel, and select Save.
5. Copy the webhook URL the dialog shows. It is on logic.azure.com or …api.powerplatform.com and contains its own signature (sig=), so treat it as a secret.
6. In UpButler, open Alerts, New channel, Microsoft Teams, paste the URL and press Send test.

curl -X POST https://upbutler.com/api/v1/channels \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "Teams #ops", "type": "teams", "url": "https://prod-00.westeurope.logic.azure.com:443/workflows/…"}'

UpButler posts one Adaptive Card (schema 1.4) in the envelope the template expects. Buttons are Action.OpenUrl, because a Workflows webhook cannot receive button callbacks; each opens its confirm page.

{
  "type": "message",
  "summary": "🔴 Checkout is failing for users since 14:02",
  "attachments": [{
    "contentType": "application/vnd.microsoft.card.adaptive",
    "contentUrl": null,
    "content": {
      "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
      "type": "AdaptiveCard",
      "version": "1.4",
      "msteams": { "width": "Full" },
      "body": [
        { "type": "TextBlock", "text": "🔴 Checkout is failing for users since 14:02 (4 minutes after deploy abc1234 by Vercel).", "weight": "Bolder", "size": "Medium", "wrap": true, "color": "attention" },
        { "type": "TextBlock", "text": "Checkout is DOWN", "wrap": true, "isSubtle": true },
        { "type": "FactSet", "facts": [{ "title": "Target", "value": "https://acme.com/checkout" }, { "title": "Error", "value": "HTTP 500" }] }
      ],
      "actions": [
        { "type": "Action.OpenUrl", "title": "Roll back", "url": "https://upbutler.com/act/…", "style": "destructive" },
        { "type": "Action.OpenUrl", "title": "Acknowledge", "url": "https://upbutler.com/ack/…" },
        { "type": "Action.OpenUrl", "title": "Open in UpButler", "url": "https://upbutler.com/app/incidents/inc_…" }
      ]
    }
  }]
}

Microsoft's documentation: Create an incoming webhook (Workflows), Send messages in Teams using incoming webhooks, Retirement of Office 365 connectors within Microsoft Teams, the Teams webhook trigger.

#PagerDuty

Available on the Starter plan and up. UpButler uses the Events API v2, so all it needs is the integration key of one service.

1. In PagerDuty, open Services, pick the service, then the Integrations tab.
2. Select Add an integration, choose Events API V2 and add it.
3. Open the new integration and copy its Integration Key (32 characters).
4. In UpButler, open Alerts, New channel, PagerDuty, paste the key and choose the region of your PagerDuty account (US or EU). Send test opens an informational alert and resolves it straight away.

curl -X POST https://upbutler.com/api/v1/channels \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "PagerDuty", "type": "pagerduty", "integrationKey": "<32-character integration key>", "region": "us"}'

#How alerts stay in sync

A PagerDuty channel always receives the acknowledge and resolve events, whatever its event filter, so an alert it opened is never left open. Severity: a down monitor is critical; manual incidents map impact critical, major, minor to critical, error, warning. The team's AI analysis rides along in custom_details, and every alert action is a link on the PagerDuty incident.

{
  "routing_key": "<integration key>",
  "event_action": "trigger",
  "dedup_key": "inc_0n3qc1a2b3c4d5e6f7g",
  "client": "UpButler",
  "client_url": "https://upbutler.com/app/incidents/inc_0n3qc1a2b3c4d5e6f7g",
  "links": [
    { "href": "https://upbutler.com/act/…", "text": "Roll back" },
    { "href": "https://upbutler.com/ack/…", "text": "Acknowledge" },
    { "href": "https://upbutler.com/app/incidents/inc_…", "text": "Open in UpButler" }
  ],
  "payload": {
    "summary": "Checkout is failing for users since 14:02 (4 minutes after deploy abc1234 by Vercel).",
    "source": "https://acme.com/checkout",
    "severity": "critical",
    "timestamp": "2026-10-10T11:02:31.000Z",
    "component": "Checkout",
    "class": "monitor.down",
    "custom_details": {
      "What happened": "Checkout is failing for users since 14:02 (4 minutes after deploy abc1234 by Vercel).",
      "Summary": "HTTP 500",
      "Target": "https://acme.com/checkout",
      "AI analysis": "The 500s started four minutes after deploy abc1234 …",
      "Event": "monitor.down",
      "Incident id": "inc_0n3qc1a2b3c4d5e6f7g"
    }
  }
}

#Two-way sync (optional)

To acknowledge or resolve in PagerDuty and have the incident follow in UpButler, add a v3 webhook subscription:

1. Open the channel's edit form in UpButler and copy its webhook URL (https://upbutler.com/hooks/pagerduty/ch_…, also config.webhookUrl in the API).
2. In PagerDuty, go to Integrations, Generic Webhooks (v3), New Webhook.
3. Paste the URL, set Scope to the same service, and tick the events incident.acknowledged and incident.resolved.
4. Save, copy the signing secret PagerDuty shows once, and paste it into the channel (or send it with PATCH /channels/:id).

curl -X PATCH https://upbutler.com/api/v1/channels/ch_… \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"webhookSecret": "<secret PagerDuty showed when you created the subscription>"}'

• Every delivery is checked against X-PagerDuty-Signature (v1= HMAC-SHA256 of the raw body, as PagerDuty documents). Without a stored secret, or with a wrong signature, the endpoint answers 401 and does nothing.
• Acknowledged in PagerDuty: the incident is acknowledged here by “name (via PagerDuty)”, which stops reminders and escalation.
• Resolved in PagerDuty: the incident is resolved here, unless one of its monitors is still failing. Then it stays open with a timeline note, so a status page never says “resolved” while checks fail.
• Only alerts UpButler opened are touched (the PagerDuty incident_key must be an incident of the channel's workspace).

#Opsgenie and Jira Service Management

Available on the Starter plan and up.

1. Jira Service Management: open Operations, then your team, Integrations, Add integration, search for API, name it and turn it on. Opsgenie: Teams, your team, Integrations, Add integration, API.
2. Copy the integration's API key. It needs create-and-update access (the default).
3. In UpButler, open Alerts, New channel, Opsgenie, paste the key and choose the product: Jira Service Management, Opsgenie (US) or Opsgenie (EU). Send test creates a P5 alert and closes it.

curl -X POST https://upbutler.com/api/v1/channels \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "JSM on-call", "type": "opsgenie", "integrationKey": "<API key of the API integration>", "region": "jsm"}'

The alert's alias is the UpButler incident id, so a repeat is de-duplicated and the alert is acknowledged and closed by alias in step with the incident, with the same mapping as PagerDuty. Priority: down monitors and critical incidents are P1, major P2, minor and degraded P3.

POST https://api.atlassian.com/jsm/ops/integration/v2/alerts      (region "jsm")
POST https://api.opsgenie.com/v2/alerts                           (region "us")
POST https://api.eu.opsgenie.com/v2/alerts                        (region "eu")
Authorization: GenieKey <API key>

{
  "message": "Checkout is failing for users since 14:02",
  "alias": "inc_0n3qc1a2b3c4d5e6f7g",
  "description": "Checkout is failing for users since 14:02 (4 minutes after deploy abc1234 by Vercel).\nCheckout is DOWN\nHTTP 500\nTarget: https://acme.com/checkout\n\nRoll back: https://upbutler.com/act/…\nAcknowledge: https://upbutler.com/ack/…",
  "priority": "P1",
  "source": "UpButler",
  "entity": "Checkout",
  "tags": ["upbutler", "monitor.down"],
  "details": { "Target": "https://acme.com/checkout", "Incident id": "inc_…", "Roll back": "https://upbutler.com/act/…" }
}

# later, in step with the incident
POST …/alerts/inc_0n3qc1a2b3c4d5e6f7g/acknowledge?identifierType=alias
POST …/alerts/inc_0n3qc1a2b3c4d5e6f7g/close?identifierType=alias

Atlassian's documentation: Opsgenie migration and end of support, Jira Service Management Operations: integration events API, Opsgenie Alert API.

#Push notifications

UpButler is an installable web app. A member turns push on per device, and from then on alerts arrive as system notifications with action buttons, even when no UpButler tab is open. No app store, and on every plan.

1. On a phone, install it first. Android (Chrome): menu, Add to Home screen / Install app. iPhone and iPad (iOS 16.4 or later, Safari): Share, Add to Home Screen, then open UpButler from the icon. On a desktop browser nothing needs installing.
2. Open Alerts (or Settings → Contact details) and press Enable push on this device. Allow notifications when the browser asks, then Send test.
3. Create a Push channel (Alerts, New channel, Push) so alerts are sent to devices. It notifies every member with push enabled, or only the members you tick.

curl -X POST https://upbutler.com/api/v1/channels \
  -H "Authorization: Bearer $UPBUTLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "Phones", "type": "push"}'

• Buttons. The notification shows the alert's first actions (browsers allow two, some three) in the usual order: Roll back, Let Claude fix, Acknowledge, Stop run, Mute 1h. A button opens its signed confirm page; tapping the notification opens the incident. Links are signed for the member who receives them, so an acknowledgement names that person.
• On-call. When an escalation policy pages a person, or the on-call person of a schedule, every device they enabled is notified next to the email. Nothing to configure in the policy.
• Devices. Up to 10 per member, listed where you enabled them, each removable. When a browser's subscription expires the push service says so (404 or 410) and UpButler removes the device.
• Privacy. Messages are encrypted for the device (RFC 8291); the push service of the browser vendor relays them but cannot read them.

{
  "title": "🔴 Checkout is failing for users since 14:02",
  "body": "HTTP 500",
  "url": "https://upbutler.com/app/incidents/inc_…",
  "tag": "inc_0n3qc1a2b3c4d5e6f7g",
  "icon": "/brand/mark-192.png",
  "badge": "/brand/mark-64.png",
  "requireInteraction": true,
  "actions": [
    { "action": "rollback", "title": "Roll back", "url": "https://upbutler.com/act/…" },
    { "action": "ack", "title": "Acknowledge", "url": "https://upbutler.com/ack/…" }
  ]
}

#Self-hosting: VAPID keys

Push services identify the sender by a key pair (VAPID, RFC 8292). UpButler reads VAPID_PUBLIC_KEY and VAPID_PRIVATE_KEY from the environment; when they are missing it generates a pair on first use and keeps it in the database (the private half encrypted with APP_SECRET), so there is no manual step. Changing the keys invalidates every existing device subscription; members enable push again.

# optional: UpButler generates and stores a pair on first use when these are not set
VAPID_PUBLIC_KEY=<65-byte P-256 public key, base64url>
VAPID_PRIVATE_KEY=<32-byte private key, base64url>
VAPID_SUBJECT=mailto:[email protected]