Back to Cookbook

Nextflow Permissions & File-Lock Troubleshooter

Fix container UID mismatches and lock-unsupported filesystems.

Diagnose and resolve Nextflow failures caused by permission errors inside containers or by filesystems that do not support required file locks for Nextflow's cache DB.

CommunitySubmitted by CommunityWork12 min

INGREDIENTS

🐙GitHub🔍Web

PROMPT

You are OpenClaw. Ask for nextflow.config (executor + container settings), filesystem type for the work dir, and the exact error. Then propose: (1) a permission/UID strategy for containers and (2) a workDir/launchDir placement strategy to ensure file-lock support. Provide a minimal "hello world" Nextflow test to validate.

Pain point

Nextflow pipelines fail with permission denied (often in Docker/Singularity contexts), or with cache DB errors

indicating file locks aren't supported.

Repro/diagnostic steps

  1. Capture failing process log (.command.*), executor (local/SLURM/K8s), and container engine.
  2. Identify filesystem type for launchDir/workDir (NFS, Lustre, object-store mounts, etc.).

Root causes (common)

  • Container user UID/GID differs from host, so work dir is not writable.
  • Nextflow executed on shared FS lacking lock support (cache DB cannot open).
  • Misplaced workDir/launchDir on constrained storage.

Fix workflow

  1. Ensure work directories are writable by the container user (UID mapping or chmod strategy).
  2. Move launchDir to a filesystem supporting file locks; set workDir separately if needed.
  3. Re-run with a minimal pipeline to validate.

Expected result

  • Nextflow can create and use its cache DB; task working dirs are writable in containers.

References

  • https://github.com/nextflow-io/nextflow/issues/1295
  • https://stackoverflow.com/questions/59791866/filenotfoundexception-permission-denied-with-nextflow-and-docker
  • https://github.com/nextflow-io/nextflow/issues/1544
Tags:#nextflow#containers#hpc#reproducibility