KiloClawPowered by OpenClaw
Back to Cookbook
OpenClaw recipe

Flaky Test Quarantine

Stop flaky tests from blocking PRs while you fix them

Quarantine known flaky tests into a separate lane so they don't block merges, while preserving visibility and enforcing a fix-by deadline.

CommunitySubmitted by CommunityWork15 min
Try in KiloClawFree 7-day trial

INTEGRATIONS NEEDED

🐙GitHub

PROMPT

Create a skill called "Flaky Test Quarantine". The skill must: - Define how to identify flaky tests (reruns + fingerprints) - Propose the quarantine mechanism for our test framework - Specify CI changes: separate lane, reporting, and non-blocking behavior - Provide a tracking template (owner, SLA, exit criteria) Output in Markdown plus example snippets (pseudo-code acceptable).

How It Works

Flaky tests destroy trust and waste developer time. This recipe sets up a quarantine

pattern: detect → tag → isolate → fix → unquarantine.

Triggers

  • Tests fail intermittently with no code changes
  • Developers rerun CI until it passes
  • PR merges are blocked by "random red" failures

Steps

  1. Identify flaky tests by rerun history and failure fingerprints.
  2. Tag tests as flaky (framework tag/annotation) and route them to a separate CI stage.
  3. Make PR gating depend on non-flaky suites; still run flaky lane for signal.
  4. Track flaky tests in a single dashboard/issue with owners and deadlines.
  5. Require a root-cause fix (not just retries) to remove from quarantine.

Expected Outcome

  • PR pipeline regains trustworthiness.
  • Flakiness becomes bounded work with ownership, not background chaos.

Example Inputs

  • "These 8 tests fail randomly on main."
  • "E2E tests flake due to timing/race conditions."
  • "Network-dependent tests fail in CI only."

Tips

  • Quarantine is a temporary containment strategy, not acceptance of flakiness.
Tags:#flaky-tests#testing#ci-cd#developer-productivity

Related Recipes

Merge Conflict Resolver

Resolve conflicts safely and reduce repeat occurrences

Provide a consistent workflow for resolving merge conflicts and preventing recurring conflicts via smaller changes, frequent integration, and safer branching practices.

Work10 min

Release Checklist Automation

Replace manual release rituals with a reliable pipeline

Turn error-prone manual release work (version bumps, changelog updates, tagging, publishing) into an automated, auditable workflow with human approval gates.

Work25 min

Hook Lab

Improve retention with scroll-stopping openings

Generates and rewrites hooks optimized for your platform and audience intent. It produces multiple hook styles (curiosity, contrast, payoff-first) and recommends which to test, based on your goals and content type.

Creative3 min

Source Hunter

Real sources, named experts, actual quotes

Deep research that finds primary sources with named individuals, community sentiment from Reddit/HN/X, and news coverage. No summaries of summaries — actual quotes with URLs.

Creative1 min