I can't quite decide which way it is.
In clusterFunctionsSlurm.R, the getClusters function is defined which looks for a resource named cluster (singular).
https://github.com/mllg/batchtools/blob/1196047ed5115d54bde2923848c1f3ec11fda6d2/R/clusterFunctionsSlurm.R#L38-L42
The documentation of submitJobs refers to clusters (plural):
https://github.com/mllg/batchtools/blob/1196047ed5115d54bde2923848c1f3ec11fda6d2/R/submitJobs.R#L25
The problem:
if this resource / getClusters is NULL, there's no cluster specified and functions like findRunning() etc. will return an empty table despite jobs are running.
For months I was wondering why my queued Slurm jobs where listed as expired and I assumed I was doing something wrong or my template was outdated.
Took me a moment to figure out that I was accidentally correctly misspecifying clusters in my resources = list(...) call 🥴
EDIT: Oh, and the template I'm using relies on resources$clusters.
Suggested fix
I'm not sure. I was about to prepare a PR when I realized that I don't want to
- change the internal
getClusters function to look for clusters instead of cluster because that will silently break peoples existing batchtools configs and they might not notice.
- change the docs for
submitJobs to (wrongly?) suggest that the resource should be named cluster, implying that multiple clusters can not be specified (which I never tried)
At the very least I thought about introducing an assertion on provided resources to shield against this sort of thing, but I'm not sure how to go about that yet.
I can't quite decide which way it is.
In
clusterFunctionsSlurm.R, thegetClustersfunction is defined which looks for a resource namedcluster(singular).https://github.com/mllg/batchtools/blob/1196047ed5115d54bde2923848c1f3ec11fda6d2/R/clusterFunctionsSlurm.R#L38-L42
The documentation of
submitJobsrefers toclusters(plural):https://github.com/mllg/batchtools/blob/1196047ed5115d54bde2923848c1f3ec11fda6d2/R/submitJobs.R#L25
The problem:
if this resource /
getClustersisNULL, there's no cluster specified and functions likefindRunning()etc. will return an empty table despite jobs are running.For months I was wondering why my queued Slurm jobs where listed as expired and I assumed I was doing something wrong or my template was outdated.
Took me a moment to figure out that I was accidentally correctly misspecifying
clustersin myresources = list(...)call 🥴EDIT: Oh, and the template I'm using relies on
resources$clusters.Suggested fix
I'm not sure. I was about to prepare a PR when I realized that I don't want to
getClustersfunction to look forclustersinstead ofclusterbecause that will silently break peoples existing batchtools configs and they might not notice.submitJobsto (wrongly?) suggest that the resource should be namedcluster, implying that multiple clusters can not be specified (which I never tried)At the very least I thought about introducing an assertion on provided
resourcesto shield against this sort of thing, but I'm not sure how to go about that yet.