excessive memory usage.

[callumparr]

I have a paf alignment file with around 7 million alignments and a file size of just under 2Gb.

Reading in the file leads to excessive memory usage. Currently well over 30Gb of memory.

If I read in the alignment file using read.delim memory useage is a fraction of read_paf.

Is there a way to see if there is something weird about my PAF alignment file that is causing issues?

head from the alignment file

7195a176-5a05-41f7-89fb-411afdc63682    4544    381     400     +       ME_sbf1_palindrome      46      27      46      19      19      18      NM:i:0  ms:i:76 AS:i:76 nn:i:0  tp:A:P  cm:i:11 s1:i:26 s2:i:0  de:f:0  rl:i:0  cg:Z:19=        cs:Z::19
7195a176-5a05-41f7-89fb-411afdc63682    4544    1310    1341    +       ME_sbf1_palindrome      46      2       36      26      36      1       NM:i:10 ms:i:60 AS:i:58 nn:i:0  tp:A:P  cm:i:2  s1:i:10 s2:i:0  de:f:0.2353     rl:i:0  cg:Z:4=1I1=2D2=1D3=2D5=1X5=1I2=2X4=     cs:Z::4+g:1-tt:2-a:3-at:5*gc:5+g:2*ta*gc:4
7195a176-5a05-41f7-89fb-411afdc63682    4544    2962    2993    +       ME_sbf1_palindrome      46      16      45      23      32      1       NM:i:9  ms:i:53 AS:i:52 nn:i:0  tp:A:P  cm:i:2  s1:i:10 s2:i:0  de:f:0.2581     rl:i:0  cg:Z:3=1X1=1X2=2I1X7=1I2=1D2=1X1=1X5=   cs:Z::3*cg:1*tg:2+ct*at:7+c:2-t:2*at:1*ga:5