Back to Cookbook

Schema Sentinel

Get alerted when upstream schemas change before your pipelines break

Monitors your source databases, APIs, and third-party integrations for schema changes — new columns, removed columns, type changes, enum value additions — and alerts you before the downstream breakage happens.

CommunitySubmitted by CommunityWork5 min setup

INGREDIENTS

💬Slack✈️Telegram🐙GitHub

PROMPT

Create a skill called "Schema Sentinel". I'll configure it with connections to my source databases and/or APIs. On a schedule I define, the skill should: (1) Snapshot the current schema of each monitored source (tables, columns, types, constraints). (2) Compare to the previous snapshot. (3) For any changes found, classify them as: column added, column removed, column renamed (detected by type + position similarity), type changed, constraint changed, or new values in categorical columns. (4) Assess downstream impact by tracing which dbt models, views, and dashboards reference the changed columns. (5) Generate migration code if applicable. (6) Alert me via Slack or Telegram with severity (critical for removals/renames, warning for additions, info for new enum values). Maintain a full change log with timestamps.

How It Works

A vendor renames a column. A Salesforce admin adds a custom field. An API

bumps its version. You find out when everything downstream breaks at 2am.

This skill detects changes at the source so you can respond proactively.

What You Get

  • Continuous monitoring of source database schemas
  • Column-level change detection (additions, removals, renames, type changes)
  • Enum/value set change detection (new categories appearing in categorical columns)
  • Impact analysis (which downstream models, dashboards, and reports are affected)
  • Auto-generated migration code (ALTER TABLE, dbt model updates)
  • Alerts via Slack/Telegram with severity levels

Setup Steps

  1. Ask your Claw to create a "Schema Sentinel" skill with the prompt below
  2. Connect it to your source databases and critical APIs
  3. Set the monitoring frequency (hourly, daily, etc.)
  4. Configure alert channels
  5. Optionally provide your dbt project for impact analysis

Tips

  • Monitor source systems, not your own warehouse — catch changes at the origin
  • The impact analysis is crucial: "column removed → 3 dbt models affected → 2 dashboards broken"
  • Keep a schema change log for post-mortems and vendor conversations
  • Combine with data contracts if you have formal agreements with data producers
Tags:#data-quality#schema#monitoring#data-engineering