Skip to content

EpiModel/slurmworkflow_tester

Repository files navigation

Test Workflow for slurmworkflow

This repo creates a workflow testing slurmworkflow's capabilities on an HPC.

How To Use

  • Create a new GitHub repository by clicking the green Use this template button above.
  • Clone this new repo on your computer AND on the HPC (git is required on the HPC)
  • On your local computer:
    • Setup renv
      • renv::init(bare = TRUE) - then restart R
      • Install packages: renv::install(c("future.apply", "EpiModel/slurmworkflow"))
      • renv::snapshot()
    • Edit the "0-settings.R" script according to your setup
    • Commit your changes and push to GitHub (including the renv.lock file)
    • Run the script "1-make_workflow.R"

Send the generated workflow directory "workflows/test_slurmworkflow" to the HPC with:

for Windows users:

# bash - local (for WINDOWS users)
set DISPLAY=
scp -r workflows\test_slurmworkflow <user>@<hpc address>:<path_to_project_on_HPC>/workflows/

for Linux / MacOS users

# bash - local (for Linux / MacOS users)
scp -r workflows/test_slurmworkflow <user>@<hpc address>:<path_to_project_on_HPC>/workflows/
  • On the HPC
    • cd into your project directory
    • execute the workflow with:
# bash - HPC
./workflows/test_slurmworkflow/start_workflow.sh

The workflow will not work if you source the file (with source <script> or . <script>).

Note:

If you are a Windows user, the script may not be executable. You can solve it with the following command:

# bash - hpc
chmod +x workflows/test_slurmworkflow/start_workflow.sh

Then run:

# bash - HPC
./workflows/test_slurmworkflow/start_workflow.sh

Expected results

If all goes well, you should receive an e-mail a few minutes later with object looking like:

[External] Slurm Job_id=14940582 Name=test_slurmworkflow_step5 Ended, Run time 00:00:02, COMPLETED, ExitCode 0

If the ExitCode is anything other than 0, it means it failed.

On the HPC you can run ls workflows/test_slurmworkflow/log/. The folder should contain:

  • 1 "test_slurmworkflow_controler.out" file
  • 1 "test_slurmworkflow_step1_xxx_xxx.out" file
  • 1 "test_slurmworkflow_step2_xxx_xxx.out" file
  • 2 "test_slurmworkflow_step3_xxx_xxx.out" file
  • 5 "test_slurmworkflow_step4_xxx_xxx.out" file
  • 1 "test_slurmworkflow_step5_xxx_xxx.out" file

If so: SUCCESS! slurmworkflow is working correctly

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages