Back to Cookbook

Alert Tuner

Turn 500 daily alerts into the 5 that actually matter

Analyzes your alerting rules and history to find the noise — duplicate alerts, overly sensitive thresholds, alerts that never lead to action. Generates tuned configurations that keep the signal and kill the spam.

House RecipeWork5 min

INGREDIENTS

💬Slack✈️Telegram

PROMPT

Create a skill called "Alert Tuner". When I provide alert rules (Prometheus alerting rules YAML, Datadog monitor definitions, or similar) and optionally alert firing history, analyze the alerting configuration for noise: 1. Identify alerts that fire frequently but auto-resolve without action 2. Find duplicate or overlapping alerts (same root cause, multiple pages) 3. Flag thresholds that are too sensitive based on normal variance 4. Suggest alert grouping and correlation rules 5. Recommend severity re-classification 6. Generate updated alert configurations with the improvements applied For each recommendation, explain the reasoning and the expected reduction in alert volume. Output the tuned configs in the same format as the input.

How It Works

Export your alert rules and firing history and this skill finds the patterns

that waste on-call time. It identifies alerts that always auto-resolve,

duplicates firing for the same root cause, thresholds set too tight, and

missing correlation rules.

What You Get

  • Alert noise analysis: frequency, auto-resolve rate, action-taken rate
  • Identification of top noise generators (alerts that fire often but never need action)
  • Threshold tuning recommendations based on historical data
  • Alert grouping suggestions to reduce duplicate pages
  • Severity re-classification (is that P1 really a P1?)
  • Updated alert-as-code configurations (Prometheus rules, PagerDuty, Datadog monitors)

Setup Steps

  1. Export your alert rules (Prometheus rules YAML, Datadog monitor JSON, etc.)
  2. Export alert firing history if available (Prometheus ALERTS metric, PagerDuty incidents export)
  3. Ask your Claw to analyze and tune
  4. Review suggested changes and apply selectively

Tips

  • Start by tackling the top 10 noisiest alerts — that's usually 80% of the spam
  • Don't just delete noisy alerts — sometimes they need tuning, not removal
  • Use the severity re-classification to fix the "everything is P1" problem
  • Run the analysis monthly to keep alert quality high as infrastructure changes
Tags:#monitoring#alerting#on-call#devops