[GavinDPinto]

Closes #30. Closes #31. Part of #22.

watcloud slurm run builds the Slurm request from plain flags, prints it, and runs it.

watcloud slurm run --gpu-mem 8G --disk 40G --mem 16G --time 4h --docker
watcloud slurm run --batch --gpu-mem 8G --time 12h -- python train.py

What it does

• Flags to srun/sbatch: --cpus, --mem, --disk (tmpdisk), --gpu-mem (shard), --gpus/--gpu-type, --time, --partition. --dry-run prints the command instead.
• --docker starts dockerd in the job and sets DOCKER_HOST. It defaults --disk to 20 GiB instead of the silent 100 MiB.
• --batch submits with sbatch: a command through --wrap, or a script file whose #SBATCH lines still apply. It allows compute_dense, requires --time, and prints the job ID, output file and a watcloud subscription hint.
• Checks before submitting:
  • Sizes and times need a unit.
  • Requests no node could ever run are rejected, naming the limit hit (read from sinfo).
  • The rules in job_submit.lua are applied up front: no interactive jobs on compute_dense, a warning past 6 h.
  • If Slurm doesn't answer within 5 s, it warns that the checks were skipped.
• watcloud docker start -h now points to slurm run --docker.

Testing

• Unit tests in internal/slurm: parsing, the srun/sbatch lines, node checks against real sinfo output, and the sinfo timeout.
• 35 use cases run on the cluster on 9 Nov are written up in this comment. Highlights:
  • A CUDA image pull fails with raw srun's 100 MiB default and works with --docker.
  • A 6.14 GB PyTorch image fits in the 20 GiB default.
  • 13 common mistakes are stopped before submitting.
  • Failed, timed-out and queued batch jobs are reported the same way as with plain sbatch.

🤖 Generated with Claude Code

[GavinDPinto]

slurm-run-use-case-testing.md

The above doc has some information on the common use-cases for the command as well as some incorrect usage and the associated error messages outputted.