Processing demultiplexed UCE fastq files from NCBI SRA

[SaunakP25]

Hi all,
I am trying to process a bunch of published UCE fastq files that I got from the SRA platform of NCBI. I was unable to find the adapter and barcode information in the provided metadata. On contacting the SRA team, they mentioned that the files are trimmed and then deposited in the database. Is there a way to go ahead with processing these fastq files to convert them into fasta without the adapter and barcode information? Any suggestions and tips in this regard would be really appreciated. Thanks a lot,

Regards,
Saunak Pal
PGR, Newcastle University, UK

[brantfaircloth]

If the adapters are trimmed, you'll still need to assemble the fastq sequences into contigs. You can do that many ways, but one way would be to organize the files as described in the tutorial (see the ATTENTION box), then proceed from that step.

[SaunakP25]

Dear Dr Faircloth, Thanks a lot for your prompt response and suggestions. I will follow the same and let you know if there are any further questions. Regards, Saunak Pal

…

________________________________ From: Brant Faircloth ***@***.***> Sent: 28 August 2025 16:47 To: faircloth-lab/phyluce ***@***.***> Cc: Saunak Pal (PGR) ***@***.***>; Author ***@***.***> Subject: Re: [faircloth-lab/phyluce] Processing demultiplexed UCE fastq files from NCBI SRA (Issue #376) ⚠ External sender. Take care when opening links or attachments. Do not provide your login details. [https://avatars.githubusercontent.com/u/54026?s=20&v=4]brantfaircloth left a comment (faircloth-lab/phyluce#376)<#376 (comment)> If the adapters are trimmed, you'll still need to assemble the fastq sequences into contigs. You can do that many ways, but one way would be to organize the files as described in the tutorial<https://phyluce.readthedocs.io/en/latest/tutorials/tutorial-1.html#assemble-the-data> (see the ATTENTION box), then proceed from that step. — Reply to this email directly, view it on GitHub<#376 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BTMK33W32CS3NWYF76O66QL3P4QA3AVCNFSM6AAAAACFB3LJVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMZUGA2DAMJZGQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[SaunakP25]

Dear Dr Faircloth,
Thanks for the suggestion. I created a contig file, but I am getting the following error.
$ phyluce_assembly_assemblo_spades \

--conf assembly.conf \
--output spades-assemblies \
--cores 8

2025-08-28 22:20:38,291 - phyluce_assembly_assemblo_spades - INFO - =========== Starting phyluce_assembly_assemblo_spades ===========
2025-08-28 22:20:38,292 - phyluce_assembly_assemblo_spades - INFO - Version: 1.7.3
2025-08-28 22:20:38,292 - phyluce_assembly_assemblo_spades - INFO - Commit: None
2025-08-28 22:20:38,292 - phyluce_assembly_assemblo_spades - INFO - Argument --config: /mnt/storage/nobackup/proj/smwmh/SphenomorphinUCEtoPRG_Phyluce/assembly.conf
2025-08-28 22:20:38,292 - phyluce_assembly_assemblo_spades - INFO - Argument --cores: 8
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Argument --dir: None
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Argument --log_path: None
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Argument --memory: 8
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Argument --no_clean: False
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Argument --output: /mnt/storage/nobackup/proj/smwmh/SphenomorphinUCEtoPRG_Phyluce/spades-assemblies
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Argument --subfolder:
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Argument --verbosity: INFO
2025-08-28 22:20:38,293 - phyluce_assembly_assemblo_spades - INFO - Getting input filenames and creating output directories
2025-08-28 22:20:38,303 - phyluce_assembly_assemblo_spades - INFO - ----------- Processing Prasinohaema_flavipes_CCA17083 -----------
2025-08-28 22:20:38,304 - phyluce_assembly_assemblo_spades - INFO - Finding fastq/fasta files
2025-08-28 22:20:38,310 - phyluce_assembly_assemblo_spades - INFO - File type is fastq
Traceback (most recent call last):
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/lib/python3.6/site-packages/phyluce/raw_reads.py", line 137, in get_input_files
if match.groups()[0] == "1":
AttributeError: 'NoneType' object has no attribute 'groups'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_assemblo_spades", line 231, in
main()
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_assemblo_spades", line 213, in main
reads = get_input_files(dir, args.subfolder, log)
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/lib/python3.6/site-packages/phyluce/raw_reads.py", line 151, in get_input_files
"The appear to be multiple files for R1/R2/Singleton reads"
OSError: The appear to be multiple files for R1/R2/Singleton reads

Since I am using pre trimmed files I am not sure how to get READ-singleton.fastq.gz file. I have also attached the config file as text doc

assembly.conf.txt

for reference. Also do I need to put each pair in respective species folders as in assembly.conf or can I put all pairs in one single folder and create config file (as in assembly2.conf.txt). Can you please let me know what is going wrong. Thanks again for all your help.

assembly2.conf.txt

[SaunakP25]

Dear Dr Faircloth, I changed the files names with R1 and R2 and it seemed to work but with errors like "CRITICAL - Expected assembly files were not found in output". In the spades-assemblies directory I got the contigs folder with respective contigs.fasta files but both are 1kb and for some reason I am unable to open these to view. I have attached the log file for your reference. Following the tutorial I tried Assembly QC but that too gave the following errors

$ for i in spades-assemblies/contigs/*.fasta;

do
phyluce_assembly_get_fasta_lengths --input $i --csv;
done
Traceback (most recent call last):
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 106, in
main()
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 62, in main
lengths = numpy.array([int(record) for record in fasta_iter(args.input)])
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 62, in
lengths = numpy.array([int(record) for record in fasta_iter(args.input)])
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 54, in fasta_iter
with open(fasta) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/storage/nobackup/proj/smwmh/SphenomorphinUCEtoPRG_Phyluce/spades-assemblies/contigs/Prasinohaema_flavipes_CCA17083.contigs.fasta'
Traceback (most recent call last):
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 106, in
main()
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 62, in main
lengths = numpy.array([int(record) for record in fasta_iter(args.input)])
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 62, in
lengths = numpy.array([int(record) for record in fasta_iter(args.input)])
File "/mnt/nfs/home/c3064051/anaconda3/envs/phyluce-1.7.3/bin/phyluce_assembly_get_fasta_lengths", line 54, in fasta_iter
with open(fasta) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/storage/nobackup/proj/smwmh/SphenomorphinUCEtoPRG_Phyluce/spades-assemblies/contigs/Prasinohaema_prehensicauda_CCA17222.contigs.fasta'

I tried running the phyluce_assembly_match_contigs_to_probes program. Please find log file attached.

Please guide me where I am going wrong and how do I resolve these errors. Thanks again,

phyluce_assembly_match_contigs_to_probes.log

phyluce_assembly_assemblo_spades.log

[brantfaircloth]

Check the spades.log file. It's likely that you ran out of RAM during assembly.