genEra could not find the raw "ncbi_lineages" file

[asifahmed23]

Hi,

I am trying to run dense and I get stuck at the GENERA step with the following error -

I am using nextlfow version 23.04.3.5875 with docker. My config file looks like -

I did not see any option in the DENSE pipeline to manually provide the path for a ncbi_lineages file.

Do you know what might be causing the issue?

Bests,
Asif

[Proginski]

Dear Asif,

I am trying to reproduce the error you encountered. It seems related to the genEra issue 24 josuebarrera/GenEra#24 . Apparently, the NCBI taxdump has changed over the last months, and produce an error when genEra generates the ncbi_lineage file.
I could fix a similar error locally by providing the old taxdump. I will add it in the README as soon as I have checked it works for you case.

Best,
Paul

[asifahmed23]

Hi Paul, thanks for your answer.

I tried running the pipeline using the old taxdump directory - by placing the taxdump directory in the same location from where I run the DENSE pipeline or by providing the path to the taxdump directory as follows -
$nextflow run i2bc/dense -profile singularity --taxdump /path_to_taxdump/taxdump -c athaliana.config

However, in both cases I get the following error -

The config file does not accept taxdump as a valid parameter. Is there a different way to provide the path to the taxdump directory manually? Can I alter this in the main pipeline script under modules/local/dense_modules.nf?

Bests,
Asif

[Proginski]

Hi Asif,

Yes you can definitely change it directly in modules/local/dense_modules.nf.
We are working on a new way the deal with the TRG definition, so in the mid-time, using the old taxdump seems a reasonable fix.
I will however give a try at the --taxdump parameter this see what's going on.
We will need to provide a link to the ancient taxdump, since it is to heavy to included in a github repo.

Best,
Paul

[asifahmed23]

I tried providing the path to the old taxdump manually, however this time I get the following error-

Due to the error, GenEra fails to produce the species specific ncbi_lineages file and thus says it can not find it. Do you have an idea what could be causing this issue?

Bests,
Asif

[Proginski]

Thank you for this try. So it seems that the simplest fix would be to provide both -r and -d arguments to genEra with valid (legacy-based) files. It works for me when running genEra alone. I now need to try inside DENSE, and to choose a way to host ncbi_lineages.csv and NEW_TAXDUMP/ so it just has to be downloaded along with the workflow.

[Proginski]

Dear Asif,

I could run genEra successfully (ouside and inside DENSE) by providing both '-d' and '-r' arguments

	genEra \
	-t $focal_taxid \
	-q $focal_CDS \
	-a $neighbors_CDS \
	-n ${task.cpus} \
	-b \${database} \
	-r ${taxdump}/ncbi_lineages_2025-06-18.csv \
	-d $taxdump

I asked the I2BC to host the legacy (fine) taxdump, along with ncbi_lineages_2025-06-18.csv so DENSE could directly download it. I'll let you know once it's done so we can test and use it.

Best,

[asifahmed23]

Dear Paul,

Thanks for giving it a try. I tried running DENSE with the -r and -d commands manually added in the dense_modules.nf. However, now I get the following error -

I am not sure what is causing this. Let me know in case you have an idea.

Note. I downloaded all the genomes and their annotation files from NCBI refseq.

Bests,
Asif

[Proginski]

Dear Asif,

I just pushed a new commit (master branch) with integrated taxdump and ncbi_lineages .
It seems there is a problem with the use of ncbitax2lin inside GenEra sometimes.
The commit works fine for me, could you tell me if it works for you ?
To start on a clean install, just remove ~/.nextflow/assets/i2bc/dense/, as well as work/taxdump.

Best,
Paul

[asifahmed23]

Dear Paul,

Thank you for updating the pipeline. When I try it now, the taxdump issue is no longer there. However, I still get the following error which I have not been able to resolve -

In case you have any idea what might be causing this, let me know.

Bests,
Asif

[Proginski]

Dear Asif,

I'm kind of struggling this this GenEra error (related to josuebarrera/GenEra#24).
I need to dig into it cause it seems to affect several DENSE users. Unfortunately, I am still the only maintainer of DENSE for the next few weeks, and it might take me some time.
Meanwhile, if you can use GenEra outside of DENSE you can totally enter DENSE directly with the option "genera_out = "GENERA_RESULTS/example_gene_ages.tsv".

Best,
Paul