DAG Builder
Describe a pipeline in English, get a working Airflow DAG
Skip the boilerplate. Describe your data pipeline in plain English and get a complete Airflow DAG with proper dependencies, error handling, retries, and alerting. Also debugs failing DAGs by analyzing task logs.
INGREDIENTS
PROMPT
Create a skill called "DAG Builder". When I describe a data pipeline in plain English (e.g., "Every day at 6am, extract data from our Postgres source, load it into Snowflake, run the dbt models, then refresh the Tableau extract"), generate a complete Airflow DAG including: (1) Proper operator imports and task definitions. (2) Task dependencies matching the described order. (3) Retry logic (default: 3 retries with 5-minute delay). (4) Timeout settings based on expected duration. (5) Failure alerting via Slack or email. (6) DAG documentation string. (7) SLA configuration if timing requirements are mentioned. When I paste a failed task log, diagnose the issue: identify the error, explain the root cause, and suggest a fix. Support Airflow 2.x TaskFlow API and classic operator syntax.
How It Works
Airflow DAG definitions are verbose — even a simple "run this script daily
at 10am" requires dozens of lines of Python boilerplate. This skill generates
the boilerplate from descriptions and debugs failures from logs.
What You Get
- Complete Airflow DAG file generated from a plain-English pipeline description
- Proper task dependencies, retries, and timeout handling
- Alert configuration (Slack, email) on failure
- SLA miss detection
- Failure diagnosis from task log analysis
- DAG documentation auto-generated
Setup Steps
- Ask your Claw to create a "DAG Builder" skill with the prompt below
- Describe your pipeline: what runs, in what order, on what schedule
- Get back a complete DAG file ready to deploy
- For failing DAGs, paste the task log for diagnosis
Tips
- Include your Airflow version — the API changes between major versions
- The generated DAGs include reasonable defaults for retries, timeouts, and pools
- For simple needs (run a script daily), consider if GitHub Actions might be simpler
- The failure diagnosis works with Airflow, Dagster, and Prefect logs