Back to Cookbook

Snakemake Unlock & Incomplete-Run Recovery

Resume safely after crashes without corrupting outputs.

Resolve Snakemake LockException, unlock safely, and recover from partial outputs after kill signals or interrupted jobs.

CommunitySubmitted by CommunityWork6 min

INGREDIENTS

🐙GitHub🔍Web

PROMPT

You are OpenClaw. Ask for the Snakemake command used, the directory structure, and whether multiple runs might overlap. Provide a safe unlock + rerun-incomplete sequence, plus a short policy for structuring outdirs per run/config so locks don't recur.

Pain point

Snakemake refuses to run because the working directory "cannot be locked," often after an interrupted run.

Repro/diagnostic steps

  1. Capture the LockException message and working directory path.
  2. Verify no other Snakemake process is running on the same target outputs.

Root causes (common)

  • Stale lock files left after kill/power loss.
  • Concurrent runs targeting overlapping outputs.

Fix workflow

  1. Confirm no active Snakemake instance is running.
  2. Use `--unlock` to remove stale locks.
  3. Re-run with `--rerun-incomplete` if partial outputs exist.
  4. Add guarded patterns to avoid two runs writing the same outputs.

Expected result

  • Workflow resumes; incomplete outputs are rebuilt deterministically.

References

  • https://stackoverflow.com/questions/59642199/how-can-i-run-multiple-runs-of-pipeline-with-different-config-files-issue-with
  • https://github.com/snakemake/snakemake/blob/main/src/snakemake/exceptions.py
Tags:#workflow#snakemake#reproducibility#hpc