java.lang.OutOfMemoryError: Java heap space with FastQC

[jlandry69]

Description of the bug

Hi all,
I am running the pipeline on nanopore and PacBio long read data and having the same error as described here:

"java.lang.OutOfMemoryError: Java heap space" with FastQC v0.11.9

The latest version of fastqc can now handle this with the "--memory" argument.

Would it be possible ti update the fastqc version to the latest please?

Many thanks

Command used and terminal output

No response

Relevant files

No response

System information

No response

[eit-maxlcummins]

I got fastqc to finish by providing a lot of memory, CPUs and providing a java environment variable to provide more memory, and patching to a newer fastqc version.

The diff below (which might look a bit enigmatic to some) basically just involves swapping the container for the fastqc module from 'biocontainers/fastqc:0.12.1--hdfd78af_0' to 'quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0' .

My fastq file was 125 GB.

I started by providing 200GB, it still wasn't working, so I updated the fastqc version, lowered it to 64, still failed, raised to 128, it finished.

My notes:

---

I found that it failed very quickly (at 5%) and was just hanging for 24 hours

Picked up JAVA_TOOL_OPTIONS: -Xmx48g
Started analysis of HG002_PAW71238_R1.gz
Approx 5% complete for HG002_PAW71238_R1.gz
Exception in thread "Thread-1" java.lang.OutOfMemoryError: Java heap space
        at uk.ac.babraham.FastQC.Modules.GCModel.GCModel.<init>(GCModel.java:69)
        at uk.ac.babraham.FastQC.Modules.PerSequenceGCContent.processSequence(PerSequenceGCContent.java:198)
        at uk.ac.babraham.FastQC.Analysis.AnalysisRunner.run(AnalysisRunner.java:89)
        at java.base/java.lang.Thread.run(Thread.java:834)

Thus I allocated more ram etc directly and via the java_tool_options

withName: 'FASTQC'         { time = '24h'; memory = '200 GB'; cpus = 12; env.JAVA_TOOL_OPTIONS='-Xmx180g' }

Still always failing. Lets try updating fastqc - found this discussion

s-andrews/FastQC#86

nf-core modules update fastqc --no-preview

Also reduced fastqc memory

    withName: 'FASTQC'         { time = '24h'; memory = '64 GB'; cpus = 12; env.JAVA_TOOL_OPTIONS='-Xmx48g' }

Then I reran:

# Run Nextflow
nextflow run main.nf \
  --input samplesheet.csv \
  --protocol DNA \
  -profile docker \
  --skip_demultiplexing \
  --skip_modification_analysis \
  --skip_xpore \
  --skip_m6anet \
  --skip_fusion_analysis \
  --skip_quantification \
  --skip_differential_analysis \
  --max_memory 200GB \
  -resume

Still wasnt working.

Modified fastqc main.nf

┃│ 1 --- modules/nf-core/fastqc/main.nf │┃
┃│ 2 +++ modules/nf-core/fastqc/main.nf │┃
┃│ 3 @@ -5,7 +5,7 @@ │┃
┃│ 4 conda "${moduleDir}/environment.yml" │┃
┃│ 5 container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? │┃
┃│ 6 'https://depot.galaxyproject.org/singularity/fastqc:0.12.1--hdfd78af_0' : │┃
┃│ 7 - 'biocontainers/fastqc:0.12.1--hdfd78af_0' }" │┃
┃│ 8 + 'quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0' }" │┃
┃│ 9 │┃
┃│ 10 input: │┃
┃│ 11 tuple val(meta), path(reads) │┃
┃│ 12 │┃
┃╰───────────────────────────

Then I ran nf-core modules patch fastqc

OK This time it got to 25% complete instead of 5%. Will now crank memory up

Unable to find image 'quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0' locally
0.12.1--hdfd78af_0: Pulling from biocontainers/fastqc
a8697f388c82: Pulling fs layer
a8697f388c82: Download complete
a8697f388c82: Pull complete
Digest: sha256:e194048df39c3145d9b4e0a14f4da20b59d59250465b6f2a9cb698445fd45900
Status: Downloaded newer image for quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0
Value "5461.3333333333" invalid for option memory (number expected)
Picked up JAVA_TOOL_OPTIONS: -Xmx48g
application/gzip
Started analysis of HG002_PAW71238_R1.gz
Approx 5% complete for HG002_PAW71238_R1.gz
Approx 10% complete for HG002_PAW71238_R1.gz
Approx 15% complete for HG002_PAW71238_R1.gz
Approx 20% complete for HG002_PAW71238_R1.gz
Approx 25% complete for HG002_PAW71238_R1.gz
Terminating due to java.lang.OutOfMemoryError: Java heap space

Increasing memory

    withName: 'FASTQC'         { time = '24h'; memory = '128 GB'; cpus = 12; env.JAVA_TOOL_OPTIONS='-Xmx100g' }