Issue with RAM

[jb23590]

Dear Brant,

Recently when running Phyluce SPAdes I have been getting a k-mer error, with varying amount of GB described. For example: 'The reads contain too many k-mers to fit into available memory. You need approx. 42.76GB of free RAM to assemble your dataset'

If I run SPAdes without Phyluce, and specify for example 80GB, the run completes this step. Does Phyluce assign a certain level of RAM to this step? How can I please make adjustments to the Phyluce pipeline in order to avoid this problem? I am encountering various issues trying to run without Phyluce, which I would like to avoid while continuing to use Phyluce.

Thank you,
Jay

[brantfaircloth]

Hi Jay,

You can adjust the RAM dedicated to the job using the --memory option, which takes an integer value (in GB RAM).

-b

[jb23590]

Hi Brant - thank you, this has worked on test samples but the bulk are still running on SPAdes so I am yet to receive that output. Have you ever tried running phyluce_assembly_assemblo_spades in parallel?

Thanks again
Jay

[brantfaircloth]

Yes, you can do that - but you'll need enough ram to make up for the parallelism (e.g. if each batch needs 80 GB RAM, then you'll need a total of 160 GB RAM). When I have lots of samples to run, I usually usually split them up into batches, then run each batch on a different node/machine.

If things are taking super long to run, then you can also downsample your reads - I'm not sure what numbers you are working with, read-wise, but we usually shoot for 4-6 M reads per sample. If we have more, we randomly downsample w/ seqtk.

[jb23590]

Wonderful - thank you Brant. All the best