Back to Cookbook

Metric Watchdog

Catch conflicting metric definitions before they reach a meeting

Scans your SQL queries, dbt models, and dashboard definitions to find every place a key metric is calculated. Flags inconsistencies — like marketing counting revenue with refunds and finance counting without — before they cause a trust-destroying meeting.

CommunitySubmitted by CommunityWork5 min setup

INGREDIENTS

🐙GitHub💬Slack

PROMPT

Create a skill called "Metric Watchdog". I'll give you the names of key business metrics (like "revenue", "churn rate", "active users", "conversion rate"). Search through my SQL files, dbt models, dashboard definitions, and any other code to find every place each metric is calculated. For each metric, show: (1) Every distinct definition found, with the source file and exact SQL/DAX/LookML. (2) A side-by-side comparison highlighting where definitions differ (filters, joins, date ranges, inclusion/exclusion criteria). (3) The likely root cause of each discrepancy. (4) A recommended canonical definition. Store canonical definitions in a registry. On subsequent runs, alert me when any new query or dashboard defines a tracked metric differently from the registry.

How It Works

Nothing erodes data trust faster than two dashboards showing different

revenue numbers. The root cause is almost always inconsistent metric

definitions scattered across dozens of queries and dashboards. This skill

finds them all and shows you the differences.

What You Get

  • Every SQL/DAX/LookML definition of a given metric across your codebase
  • Side-by-side comparison showing where definitions diverge
  • Root cause identification (e.g., "Dashboard A excludes refunds, Dashboard B doesn't")
  • A canonical metric registry you can maintain
  • Alerts when new queries define a metric differently from the registry

Setup Steps

  1. Ask your Claw to create a "Metric Watchdog" skill with the prompt below
  2. Tell it which metrics to track (revenue, churn, MAU, conversion rate, etc.)
  3. Point it at your SQL files, dbt project, and/or BI tool
  4. Get a report showing all definitions and discrepancies

Tips

  • Start with the 5 metrics that appear on your executive dashboard
  • The side-by-side comparison is powerful ammo for getting alignment in meetings
  • Set this up to run weekly and alert you when new inconsistencies appear
  • Pairs with Schema Scout for understanding which source tables each definition uses
Tags:#data-quality#metrics#governance#trust