Version 0.2.2 errors on bam file -- invalid pairs

[SamCT]

Hello,

Thank you for the tool. I was excited to see the update. However, trying to run:

samtools view -h HiC_alignment.bam | PretextMap --mapq 0 --highRes -o Organism.map

I get an error. This same command works using version 0.1.9 with my dataset.

std out:

[PretextMap status] :: Initializing working set mutex
[PretextMap status] :: Creating memory arena of size 16 GB
[PretextMap status] :: Initializing thread pool with 3 threads
[PretextMap status] :: Running in high resolution mode
[PretextMap status] :: Initializing line buffer queue
[PretextMap status] :: Initializing images
[PretextMap status] :: Creating filters
[PretextMap status] :: Reading from stdin
[PretextMap status] :: Starting input processing thread
[PretextMap status] :: Waiting for processing to complete
[PretextMap status] :: Input source: pipe/FIFO (fd=0, mode=10600)

[PretextMap error] :: Invalid pairs format: First line must start with '#' but found '0x40' (@)
[PretextMap error] :: Buffer contents at error:

0000: 40 48 44 09 56 4E 3A 31 2E 35 09 53 4F 3A 75 6E   @HD.VN:1.5.SO:un
0010: 73 6F 72 74 65 64 09 47 4F 3A 71 75 65 72 79 0A   sorted.GO:query.

Looks like it's expecting a pairs file? Has the format input changed?

Thanks,
Sam

[yumisims]

Hello Sam and @prototaxites ,

Thanks for the report. We were aware of this bug, and I have just submitted a PR with the necessary changes. Once the PR is approved and merged, we will proceed with a bug-fix release.

In the meantime, could you please try the pairfilefix branch? It has been tested and should work for both pair file and BAM file inputs. Please let me know if you need any help with the build.

Best,

Yumi

[SamCT]

Hi Yumi,

Having trouble with this build on my HPC.

Got it built but have an error:
free(): invalid pointer
/pretext_bugfix.commands: line 3: 1551089 Exit 141 samtools view -@ 32 -h Large_file.bam
1551090 Aborted (core dumped) | PretextMap -o Output_MAPFIX.map --highRes --mapq 0
Command exited with non-zero status 134

Not sure if this is an issue with my build or if its something else. I may have to wait for the PR push :)

Thanks,
sam

[yumisims]

Hi Sam,

It looks like there might be an issue with the data being piped to PretextMap.

Could you please try running the command directly: samtools view -h Large_file.bam | PretextMap -o Output_MAPFIX.pretext --highRes --mapq 0

Alternatively, could you try the bamfix branch? It contains specific memory management updates for BAM input. Please let me know the outcome of these tests.

Thank you, Yumi

[SamCT]

Hi Yumi,

Yes, apologies, the error above was with the bamfix branch. Which is why I'm not sure if the build completed correctly.

samtools view -h Large_file.bam | PretextMap -o Output_MAPFIX.pretext --highRes --mapq 0
[PretextMap status] :: Running with AVX2 CPU extensions
[PretextMap status] :: Initializing working set mutex
[PretextMap status] :: Creating memory arena of size 16 GB
[PretextMap status] :: Initializing thread pool with 3 threads
[PretextMap status] :: Running in high resolution mode
[PretextMap status] :: Initializing line buffer queue
[PretextMap status] :: Initializing images
[PretextMap status] :: Creating filters
free(): invalid pointer
Aborted (core dumped)

With 0.2.2 the error is:

samtools view -h Large_file.bam | PretextMap -o Output_MAPFIX.pretext --highRes --mapq 0
[PretextMap status] :: Running with AVX2 CPU extensions
[PretextMap status] :: Initializing working set mutex
[PretextMap status] :: Creating memory arena of size 16 GB
[PretextMap status] :: Initializing thread pool with 3 threads
[PretextMap status] :: Running in high resolution mode
[PretextMap status] :: Initializing line buffer queue
[PretextMap status] :: Initializing images
[PretextMap status] :: Creating filters
[PretextMap status] :: Reading from stdin
[PretextMap status] :: Starting input processing thread
[PretextMap status] :: Waiting for processing to complete
[PretextMap status] :: Input source: pipe/FIFO (fd=0, mode=10600)
[PretextMap error] :: Invalid pairs format: First line must start with '#' but found '0x40' (@)
[PretextMap error] :: Buffer contents at error:

0000: 40 48 44 09 56 4E 3A 31 2E 35 09 53 4F 3A 75 6E   @HD.VN:1.5.SO:un
0010: 73 6F 72 74 65 64 09 47 4F 3A 71 75 65 72 79 0A   sorted.GO:query.

Thanks,
Sam

[yumisims]

Hi Sam,

Thanks for testing that. I haven't been able to reproduce the error you’re seeing on the bamfix branch.

Would it be possible for me to have access to the BAM file you are testing? If not, could you please try testing with this standard dataset using the bamfix branch?

Test File: test.rna.paired_end.sorted.bam

Command: samtools view -h test.rna.paired_end.sorted.bam | PretextMap -o test.pretext

This will help me determine if the issue is specific to the data or the environment.

Many thanks, Yumi

[SamCT]

Hi Yumi,

Thanks for the quick feedback and working with me on this. I think there is an issue with the build on my end. I had to do some custom configuring due to our HPC wall clock being misconfigured, then I copied the PretextMap files into the subsequent conda environment to overwrite previous wrappers. Probably not the best approach.

I still get the same error with that test file. I can provide the bam file, but it is very large. I can break it down to 1Gb and send? Otherwise it is ~200 Gb. I am also OK with waiting until the fix is pushed to main and I can reinstall through the bioconda repo. Please let me know your thoughts.

Thanks,
Sam

[yumisims]

Hi Sam,

It is also possible that this is a build or environment-specific issue. Would you mind running one final test to isolate the bottleneck?

Please try: samtools view -h Large_file.bam >/dev/null

This will help us confirm if samtools can process the file successfully on its own. Thank you for your patience.

Yumi

[SamCT]

Hi Yumi,

Can confirm, no errors running samtools view -h Large_file.bam >/dev/null

Thanks,
Sam

[yumisims]

Hi Sam,

The new release (v0.2.3) is now available via Bioconda. Please let me know how it goes when you have a chance to test it.

Much appreciated,
Yumi