-
Notifications
You must be signed in to change notification settings - Fork 143
Description
Hey folks, I'm liking Iris so far. I'd like to check receptiveness to this change - I'm happy to put up a PR for this, as it would help us.
Problem
The AlertManager integration doesn't currently support the "override incident title" and "plan dynamic targets" features.
AlertManager sends a payload in this (slightly redacted) format:
{
"groupKey": "Demo system down",
"status": "firing",
"groupLabels": {"iris_plan": "page-engineer", "severity":"critical", "service":"demo-system"},
"commonLabels": {"iris_plan": "page-engineer", "severity":"critical", "service":"demo-system"},
"commonAnnotations": {"team":"demo-team", "runbook_url": "https://example.org", "description": "The service is down!"}
}
Since the groupLabels.iris_plan is set to an existing plan (page-engineer) the plan will run and the context (the above payload) will be available in the incident summary and context templates and the message templates.
Unfortunately, if we want to react to all incidents in the same way (i.e. page an on-call primary and, if they don't answer, the secondary) but want to pages to be received by different teams, we must create a plan (or potentially plans) for each team, and set the iris_plan label on each alert to be team specific (i.e. iris_plan: "page-engineer-demo-team". Ideally we'd be able to use the existing dynamic_targets feature on plans so that we only need to create one plan which could be reused by all teams.
Also, each incident has the title "alertmanager" (the name of the application). Though we can set an <h1>{{groupLabels.alertname}}</h1> in the context templates, it would be nice if this feature was supported for the AlertManager integration.
Feature request
In the AlertManager integration it'd be great if we could pull up the alertname field from groupLabels if it exists, and allow users to set dynamic targets in alert labels.
labels:
alertname: "Demo service is down"
iris_plan: page-engineer
iris_targets: '[{"role": "oncall-primary","target": "demo-team"},{"role": "oncall-secondary","target": "demo-team"}]'