Understanding parallel singer errror

[mehakmadhura]

Hii!
I am trying to use parallel singer and it returns the following error:

/singer/singer-0.1.7-beta-linux-x86_64/convert_long_ARG.py", line 118, in
main()
/singer/singer-0.1.7-beta-linux-x86_64/convert_long_ARG.py", line 114, in main
ts = read_long_ARG(node_files, branch_files, mutation_files, block_coordinates)
/singer/singer-0.1.7-beta-linux-x86_64/convert_long_ARG.py", line 73, in read_long_ARG
tables.sort()
/python3.9/site-packages/tskit/tables.py" line 2965, in sort
self._ll_tables.sort(edge_start)
_tskit.LibraryError: Right coordinate > sequence length

Can you please explain what might be wrong here.
Thanks
Mehak

[YunDeng98]

Hi @mehakmadhura I apologize for the late response, I remember seeing this issue during the conference but didn't get the time to review this carefully. What I would first recommend to do, is to switch to version 0.1.8 and if this occurs again, you can send me your output files and I will debug from there. Apologies for the trouble.

[mehakmadhura]

Hii @YunDeng98, thank you! I tried using ver8, but it did not work for me. It prints the following texts and then returns a series of errors.
Index vcf files

VCF file prefix: /netscratch/dep_coupland/grp_fulgione/mehak/data/paramask_snpable_filetered/GATK4.2_1000Genomes_chrall.filteredQ30LD5UD100K.final.b.ParaMask.SNPable.alps
Block length: 1000000

Processing segment 0
Processing segment 0
Processing segment 0
Processing segment 0
Processing segment 0
Processing segment 0
Processing segment 0
Processing segment 0
Processing segment 0
Processing segment 0
Parameters:
Effective population size: 1000000.0
Mutation rate: 7e-09
Recombination to mutation ratio: 1
Block length: 1000000
VCF file prefix: /netscratch/dep_coupland/grp_fulgione/mehak/data/paramask_snpable_filetered/GATK4.2_1000Genomes_chrall.filteredQ30LD5UD100K.final.b.ParaMask.SNPable.alps
Output file prefix: /netscratch/dep_coupland/grp_fulgione/mehak/data/singer/alps_paramask_ver8/alps_paramask_ver8
Number of MCMC samples: 10
Thinning interval length: 20
Site flip probability: 0.5
Tskit conversion freq: 1
Number of cores: 20
Indexing VCF file: /netscratch/dep_coupland/grp_fulgione/mehak/data/paramask_snpable_filetered/GATK4.2_1000Genomes_chrall.filteredQ30LD5UD100K.final.b.ParaMask.SNPable.alps.vcf with block length: 1000000

For me, I think ver 7 works better.
Thank you

[zhangming-m]

Have you solved this problem? I think my problem is same as yours.The problem I have is that my parallel_singer don't output the tree file，and I only have files like start with Quercus20_chr8_parallel_singer_0_1*(see picture)

I can convert files start with Quercus20_chr8_parallel_singer_0_1* to tree file use convert_to_tskit:
convert_to_tskit -input Quercus20_chr8_parallel_singer_0_1 -output Quercus20_chr8_parallel_singer -start 0 -end 99

This is my code:
/data/zhangmm/software/SINGER-main/SINGER/SINGER/parallel_singer -Ne 2e5 -m 1.2e-8 -vcf /data/zhangmm/Quercus20_phase/Quercus20_chr8_filtered_phased_addAA -output Quercus20_chr8_parallel_singer -n 100 -thin 20

Error：
ts = read_long_ARG(node_files, branch_files, mutation_files, block_coordinates)
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 73, in read_long_ARG
tables.sort()
File "/data/zhangmm/ENTER/lib/python3.9/site-packages/tskit/tables.py", line 3632, in sort
self._ll_tables.sort(edge_start, site_start, mutation_start)
_tskit.LibraryError: Sequence length must be > 0. (TSK_ERR_BAD_SEQUENCE_LENGTH)
Traceback (most recent call last):
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 118, in
main()
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 114, in main
ts = read_long_ARG(node_files, branch_files, mutation_files, block_coordinates)
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 73, in read_long_ARG
tables.sort()
File "/data/zhangmm/ENTER/lib/python3.9/site-packages/tskit/tables.py", line 3632, in sort
self._ll_tables.sort(edge_start, site_start, mutation_start)
_tskit.LibraryError: Sequence length must be > 0. (TSK_ERR_BAD_SEQUENCE_LENGTH)
Traceback (most recent call last):
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 118, in
main()
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 114, in main
ts = read_long_ARG(node_files, branch_files, mutation_files, block_coordinates)
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 73, in read_long_ARG
tables.sort()
File "/data/zhangmm/ENTER/lib/python3.9/site-packages/tskit/tables.py", line 3632, in sort
self._ll_tables.sort(edge_start, site_start, mutation_start)
_tskit.LibraryError: Sequence length must be > 0. (TSK_ERR_BAD_SEQUENCE_LENGTH)
Traceback (most recent call last):
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 118, in
main()
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 114, in main
ts = read_long_ARG(node_files, branch_files, mutation_files, block_coordinates)
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 73, in read_long_ARG
tables.sort()
File "/data/zhangmm/ENTER/lib/python3.9/site-packages/tskit/tables.py", line 3632, in sort
self._ll_tables.sort(edge_start, site_start, mutation_start)
_tskit.LibraryError: Sequence length must be > 0. (TSK_ERR_BAD_SEQUENCE_LENGTH)
Traceback (most recent call last):
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 118, in
main()
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 114, in main
ts = read_long_ARG(node_files, branch_files, mutation_files, block_coordinates)
File "/data/zhangmm/software/SINGER-main/SINGER/SINGER/convert_long_ARG.py", line 73, in read_long_ARG
tables.sort()
File "/data/zhangmm/ENTER/lib/python3.9/site-packages/tskit/tables.py", line 3632, in sort
self._ll_tables.sort(edge_start, site_start, mutation_start)
_tskit.LibraryError: Sequence length must be > 0. (TSK_ERR_BAD_SEQUENCE_LENGTH)
Parameters:
Effective population size: 200000.0
Mutation rate: 1.2e-08
Recombination to mutation ratio: 1
Block length: 1000000
VCF file prefix: /data/zhangmm/Quercus20_phase/Quercus20_chr8_filtered_phased_addAA
Output file prefix: Quercus20_chr8_parallel_singer
Number of MCMC samples: 100
Thinning interval length: 20
Site flip probability: 0.5
Tskit conversion freq: 1
Number of cores: 20
Indexing VCF file: /data/zhangmm/Quercus20_phase/Quercus20_chr8_filtered_phased_addAA.vcf with block length: 1000000
['parallel', '-u', '-j', '20', ':::']

[YunDeng98]

@zhangming-m thanks for the feedback. It seems that the ARGs have been successfully inferred but the conversion script fails. It should be easy to address. There should be a *.index file somewhere, can you send me a screenshot of that?

[zhangming-m]

@YunDeng98 Yes,there is indeed a Quercus20_chr8_filtered_phased_addAA.index file

[YunDeng98]

@zhangming-m there is a "convert_long_ARG.py" in the repo which you might want to use. So the usage is like "python convert_long_ARG.py vcf_prefix output_prefix iteration_index". If this does not fix your problem, you can send me the files and I can convert it for you. Just find all the vcf_prefix*_nodes_0.txt, vcf_prefix*_branches_0.txt, vcf_prefix*_muts_0.txt, vcf_prefix*_recombs_0.txt and send them to me. I can work from there.