Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Many computational endeavors benefit from some form of parallelization, and SLURM provides a way to do “embarrassingly parallel” processing relatively simply (read more about parallelization here). Additionally, submitting a SLURM array puts all parts of the job under one job ID, which simplifies canceling or otherwise managing dozens/hundreds/thousands of jobs.

Basics

Array submission works via the “--array” option of sbatch (read more about it here). This requires the start, stop, and optional step size for your array indices – each index will be given its own task ID.

...

Selecting these options without any other configuration in your script will result in the same task being submitted for each task ID.

Further configuration

As mentioned above, creating an array of tasks within a job without other configuration will create identical jobs for each task. This isn’t useless, but being able to use different data for each task may be more useful.

...