Subject: Clarification on “cluster_” identifiers and guidance on building a custom PICRUSt2 reference database

[Abelcanc3rhack3r]

Dear PICRUSt2 team,

I am currently examining the 16S rRNA reference FASTA files used in PICRUSt2 and noticed that some sequence records are labeled with identifiers such as “cluster_XXX”, while others are not.

Could you clarify what these “cluster_” identifiers represent in the context of the reference database? Specifically:

• Are these derived from OTU clustering (e.g., CD-HIT/VSEARCH)?
• Do they correspond to representative sequences from dereplicated groups?
• Or do they reflect some internal processing step specific to PICRUSt2?

In addition, I am attempting to construct a custom PICRUSt2 reference database by combining:

1. Older IMG-derived genome sequences (legacy version), and
2. The mouse gut iMGMC PICRUSt2-compatible reference dataset

My goal is to integrate both into a unified database for prediction.

Could you advise on the recommended workflow for this? In particular:

• How should sequence IDs be standardized (e.g., handling cluster vs non-cluster labels)?

• Are there specific requirements for:

• 16S sequence FASTA formatting

• Phylogenetic placement (EPA-NG / SEPP steps)

• Hidden-state prediction inputs (trait tables, marker gene copy numbers)

• Is there an existing pipeline or script to rebuild a PICRUSt2-compatible database from combined sources?

Finally, could you advise on the best point of contact for more detailed guidance on custom database construction? Should this be directed via GitHub issues, or is there a more appropriate channel?

Thank you for your time and for maintaining PICRUSt2.

Best regards,
Abel Tan

[R-Wright-1]

Hi Abel,

I'm going to answer here (rather than the email that you sent me directly) so that others will be able to look this up. GitHub issues is the most appropriate channel for this. I've answered all of your questions to the best of my ability below, but I will highlight that this is not an easy process and you could consider an alternative like filtering the GTDB reference files that we provide to only include genomes found in the mouse gut, if this is your goal. I am not familiar with the iMGMC database so don't know what the overlap with the GTDB database is. I am happy to provide help and guidance, but I am assuming in this that you are comfortable trouble-shooting bioinformatics issues for yourself and can familiarise yourself with the tools I mention. This took me several months to do for updating the reference database, annotating the genomes, etc.

The clusters that you are referring to are only in the older database and you can see full details of how they were clustered in the supplementary information for the original PICRUSt2 paper.

You can see the workflow that I followed for constructing the new database on Github here, and that is what I would recommend following for this, so that you would:

1. Combine the 16S sequences for both databases. I personally would treat the cluster vs non-cluster labels the same, but that is a decision for you.
2. I would then start from the point of "Cluster these single 16S genes for all genomes" - I used vsearch for this.
3. Align the sequences using ssu-align and check the sequence lengths.
4. Choose the best genome for each cluster. I based this on CheckM completeness, but you would need to decide whether you take one genome as the representative for each cluster or if you combine the functional predictions for all genomes in the cluster in some way and e.g. take the mean/median for each function (or see below for new predictions).
5. I was using the tree that was already constructed by GTDB for this, but you would then need to build the tree with all of the sequences that you want to use. If you use RAxML-ng to build the tree (as detailed in the original PICRUSt2 paper) then you likely won't need to do the "make the raxml_info files" step as you should already have them.
6. When you talk about the files for hidden state prediction, do you mean that you want to re-create them from scratch (i.e. re-annotating the genomes)? I used eggnog for annotation of the genomes in the new database and each genome takes a few minutes for annotation with 12 threads. So if you have tens of thousands of genomes, it will take months. If not, you could simply take the existing predictions for each of the genomes and combine them into a single table. I don't know how the annotations will compare in terms of the databases/methods used for generating them - the PICRUSt2 annotations were provided by IMG (and I struggled to find many details on how IMG created them) and I don't know how the iMGMC ones were created.
7. As mentioned, you can see the pipeline that I followed for making the new database here. I am not aware of any others, but that is not to say that someone else hasn't made one.

Best wishes,
Robyn

[Abelcanc3rhack3r]

Dear Robyn,

Thank you for your detailed and helpful response.

I attempted to install the latest PICRUSt2-SC, but I’ve run into some environment issues. My mamba installation appears to be corrupted, and when I try to create a fresh environment using conda, the solver hangs indefinitely. I am currently troubleshooting this, but it has slowed progress on testing the pipeline.

Given this, I wanted to ask for your advice on the database strategy. Based on your earlier suggestion, would it be more practical in my case to start from the GTDB-based PICRUSt2 reference and filter it to mouse gut genomes, rather than attempting to combine IMG and iMGMC into a custom database?

My primary interest is in hydrogenase-related functions, and I already have curated annotations for those (iMGMC/ IMG genomes), so I am mainly concerned with getting a reliable placement framework.

Thank you again for your guidance.

Best regards,
Abel

[R-Wright-1]

Is the installation issue when trying to install directly from conda? If so, you could try installing from source (and manually installing all of the packages if you still run into issues using the yaml file).

I'd think it will definitely be faster to go this route rather than combining multiple sources. First, I'd double-check that the functions of interest are in the GTDB database files (in the bacteria/archaea folders here).

Do you already have a list of something like NCBI genomes that are associated with the mouse gut? If you had a list of the accessions then you could simply take the subset of taxa that are in both the PICRUSt2 reference files as well as the mouse gut.

The next step would depend a little on what the placement tools are expecting, but I think you may be able to simply reduce the default bacterial/archaeal fasta files to only the genomes of interest - I'm not 100% sure on how the phylogenetic placement tools will work, but I think you may be able to just use the same tree for this but with only the taxa of interest in the reference fasta files. I'm happy to give some assistance once you get to the point of having the list of genomes of interest.

An alternative is that you just run PICRUSt2 as-is and then look at the closest genomes for each of your sequences, which is now in the standard output. This would give you an idea of how well the annotations are likely to fit your sequences and whether the genomes being used for predictions are found in the mouse gut.

I have also just seen that these is a new paper describing the GTDB database and this says that r220 was expanded with a lot of mouse gut taxa. Unfortunately I used r214 for the PICRUSt2 reference files, but I will think about updating the PICRUSt2 database to a newer GTDB version (although as I mentioned before, even if I were to start this immediately it would likely take some time).

Robyn

[Abelcanc3rhack3r]

Hi Robyn,

Thanks so much for the detailed suggestions! Given the current constraints, I think it’s best if I just stick with the default 2.6 database for now to keep things moving.

I did run into a bit of a snag while trying to set up the environment, though. When running mamba create -n picrust2 -c bioconda -c conda-forge picrust2=2.6.2, I hit a solver error regarding biom-format >=2.1.10. It seems to be a conflict between the channels that I'm currently troubleshooting.

[e1103389@CN-017 ~]$ conda activate mamba_env
(/scratch/e1103389/conda_envs/mamba_env) [e1103389@CN-017 ~]$ mamba --versiontimed out waiting for input: auto-logout

qsub: job 779287.stdct-mgmt-02 completed
[e1103389@vanda e1103389]$ conda activate mamba_env
(/scratch/e1103389/conda_envs/mamba_env) [e1103389@vanda e1103389]$ mamba create -n picrust2 -c bioconda -c conda-forge picrust2=2.6.2
warning  libmamba 'repo.anaconda.com', a commercial channel hosted by Anaconda.com, is used.

warning  libmamba Please make sure you understand Anaconda Terms of Services.

warning  libmamba See: https://legal.anaconda.com/policies/en/
bioconda/linux-64                                    5.5MB @  15.1MB/s  0.8s
bioconda/noarch                                      5.5MB @   5.0MB/s  1.4s
pkgs/main/noarch                                   827.4kB @ 345.7kB/s  0.8s
pkgs/r/linux-64                                      1.6MB @   3.2MB/s  0.8s
pkgs/main/linux-64                                  10.3MB @   4.5MB/s  3.0s
pkgs/r/noarch                                        2.0MB @   1.3MB/s  1.6s
conda-forge/noarch                                  25.3MB @   6.7MB/s  4.7s
conda-forge/linux-64                                52.1MB @   9.7MB/s  7.9s

warning  libmamba The specification of the environment does not seem solvable in your current setup.
warning  libmamba For instance, packages from different channels might be specified,
warning  libmamba whilst your current configuration might not allow their resolution.
warning  libmamba
warning  libmamba If it is the case, you need to either:
warning  libmamba  - adapt the channel ordering (e.g. by reordering the `-c` flags in your command line)
warning  libmamba  - use the flexible channel priority (e.g. using `--channel-priority flexible` in your command line)
warning  libmamba
warning  libmamba For reference, see this piece of documentation on channel priority:
warning  libmamba https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/manage-channels.html#strict-channel-priority
error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ picrust2 =2.6.2 * is not installable because it requires
       └─ biom-format >=2.1.10 *, which conflicts with any installable versions previously reported.
critical libmamba Could not solve for environment specs
(/scratch/e1103389/conda_envs/mamba_env) [e1103389@vanda e1103389]$
(/scratch/e1103389/conda_envs/mamba_env) [e1103389@vanda e1103389]$ mamba create -n picrust2 -c bioconda -c conda-forge picrust2=2.6.2
bioconda/linux-64                                           Using cache
bioconda/noarch                                             Using cache
conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache
warning  libmamba 'repo.anaconda.com', a commercial channel hosted by Anaconda.com, is used.

warning  libmamba Please make sure you understand Anaconda Terms of Services.

warning  libmamba See: https://legal.anaconda.com/policies/en/
pkgs/main/linux-64                                  10.3MB @  26.0MB/s  1.0s

warning  libmamba The specification of the environment does not seem solvable in your current setup.
warning  libmamba For instance, packages from different channels might be specified,
warning  libmamba whilst your current configuration might not allow their resolution.
warning  libmamba
warning  libmamba If it is the case, you need to either:
warning  libmamba  - adapt the channel ordering (e.g. by reordering the `-c` flags in your command line)
warning  libmamba  - use the flexible channel priority (e.g. using `--channel-priority flexible` in your command line)
warning  libmamba
warning  libmamba For reference, see this piece of documentation on channel priority:
warning  libmamba https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/manage-channels.html#strict-channel-priority
error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ picrust2 =2.6.2 * is not installable because it requires
       └─ biom-format >=2.1.10 *, which conflicts with any installable versions previously reported.
critical libmamba Could not solve for environment specs

On another note, I recall there used to be a guide on the wiki for creating a custom database using the older IMG version, but I noticed the documentation has since been updated. Would you happen to have the instructions for that older workflow or perhaps a link to an archived version of that page?

Best,
Abel

[R-Wright-1]

OK, sounds good.

Are you able to try installing the current version, v2.6.3?

What is it that you want to include in the custom database? I am not aware of there ever being instructions for creating a custom database. There are some instructions in the sequence placement step for creating custom reference files, and I have instructions for creating a custom trait table (i.e. adding a trait to the database that is not there) here. Is it one of these that you're thinking of?

Robyn