Memory issue when running taxprofiler with Kaiju

[albatalavera]

Description

A memory issue has been identified when running taxprofiler with Kaiju to analyze whole metagenomes. The process fails with the current memory allocation of 72GB. Doubling the memory to 144GB does not resolve the issue. Only when the memory is tripled to 216GB, the process completes successfully.

It is necessary to determine the appropriate memory allocation to ensure the process runs efficiently without exceeding resources.

Environment

• Tool: Taxprofiler
• Software used: Kaiju
• Data type: Whole metagenomes
• Tested memory allocations:
  • 72GB: Failed
  • 144GB: Failed
  • 216GB: Completed successfully

Tasks

1. Perform incremental memory tests to identify the minimum required allocation for successful execution.
2. Update the bu_isciii/templates/taxprofiler/DOC/taxprofiler.config file to reflect the optimal memory allocation.

[victor5lm]

This issue might not be of interest anymore since we are going to skip Kaiju when launching Taxprofiler from now on? @saramonzon

[saramonzon]

you could be interested in running kaiju if the service focus is metagenomics, so it's interesting although not really a priority :)