FASTQC command number expected for "--memory"

[ahwanpandey]

Hello,

Just wanted to report a small warning from FASTQC. I believe --memory requires integer and not decimal?

Thanks!
Ahwan

The command being run

]$ cat .command.sh
#!/usr/bin/env bash -C -e -u -o pipefail
printf "%s %s\n" 3-66130-Ascites_R1.fastq.gz 66130_Ascites_1.gz 3-66130-Ascites_R2.fastq.gz 66130_Ascites_2.gz | while read old_name new_name; do
    [ -f "${new_name}" ] || ln -s $old_name $new_name
done

fastqc \
    --quiet \
    --threads 6 \
    --memory 682.6666666667 \
    66130_Ascites_1.gz 66130_Ascites_2.gz

cat <<-END_VERSIONS > versions.yml
"DINCALCILAB_SAMURAI:SAMURAI:FASTQ_TRIM_FASTP_FASTQC:FASTQC_RAW":
    fastqc: $( fastqc --version | sed '/FastQC v/!d; s/.*v//' )
END_VERSIONS

The warning

]$ cat .command.err
INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
Value "682.6666666667" invalid for option memory (number expected)
Fontconfig error: No writable cache directories

[lbeltrame]

Eh... I've never seen this here. Do you have custom configuration for FastQC? The memory there I believe is passed from whatever is set by Nextflow.

[ahwanpandey]

Hi @lbeltrame , so the file samurai/modules/nf-core/fastqc/main.nf has the following lines of code

    // The total amount of allocated RAM by FastQC is equal to the number of threads defined (--threads) time the amount of RAM defined (--memory)
    // https://github.com/s-andrews/FastQC/blob/1faeea0412093224d7f6a07f777fad60a5650795/fastqc#L211-L222
    // Dividing the task.memory by task.cpu allows to stick to requested amount of RAM in the label
    def memory_in_mb = task.memory ? task.memory.toUnit('MB') / task.cpus : null
    // FastQC memory value allowed range (100 - 10000)
    def fastqc_memory = memory_in_mb > 10000 ? 10000 : (memory_in_mb < 100 ? 100 : memory_in_mb)

The resource configuration for FASTQC in samurai is 4Gb and 6 cpus. So when the memory_in_mb conversion happens as per the code above, the value is (4*1024)/6 = 682.6667, which is pretty much the number you see in the warning I posted above.

Again, not a big deal as the jobs seem to run without errors. Just wanted to point in out in case its something to be fixed?

BTW, samurai can take in at most 2 FASTQ files, so setting 2 cpus for FASTQC makes sense as I believe FASTQCs multithreading assigns one thread per FASTQ file

Thanks!
Ahwan

[lbeltrame]

Again, not a big deal as the jobs seem to run without errors. Just wanted to point in out in case its something to be fixed?

This comes from the upstream FastQC module (as far as I can remember), so if you have time it would be nice if you could file an issue in https://githiub.com/nf-core/modules and putting this issue in CC.
I do wonder why I never saw that warning...

[ahwanpandey]

@lbeltrame thanks for pointing me to the right direction to report this bug. FYI, when I run the pipeline by giving FASTQC 4GB and 2 cpus, there is no warning and the "gigabyte to megabyte" (i.e. memory_in_mb in the FASTQC module script) conversion produces an integer (4*1024)/2 = 2048

#!/usr/bin/env bash -C -e -u -o pipefail
printf "%s %s\n" 3-66130-Ascites_R1.fastq.gz 66130_Ascites_1.gz 3-66130-Ascites_R2.fastq.gz 66130_Ascites_2.gz | while read old_name new_name; do
    [ -f "${new_name}" ] || ln -s $old_name $new_name
done

fastqc \
    --quiet \
    --threads 2 \
    --memory 2048 \
    66130_Ascites_1.gz 66130_Ascites_2.gz

cat <<-END_VERSIONS > versions.yml
"DINCALCILAB_SAMURAI:SAMURAI:FASTQ_TRIM_FASTP_FASTQC:FASTQC_RAW":
    fastqc: $( fastqc --version | sed '/FastQC v/!d; s/.*v//' )
END_VERSIONS

[lbeltrame]

FTR, you can likely use a custom nextflow.config to do this in case you don't want t touch the modules (it's supported by nf-core).

[ahwanpandey]

@lbeltrame that is what I always do, of course :)

The warning is issued when there is a combination of CPU and MEM that produces a non-integer.