Add multithreading via rayon

[AroneyS]

Had some issues since the parse_fastx_file reader isn't an iterator for some reason, so might be a bit clumsy.

Should I add a test that actually uses multiple threads?

[wwood]

Yes please. Can you also check it is faster and this isn't for nothing? -\------------- Ben Woodcroft Group leader, Centre for Microbiome Research, QUT

…

________________________________ From: Samuel Aroney ***@***.***> Sent: Wednesday, December 11, 2024 7:59:31 AM To: wwood/smafa ***@***.***> Cc: Ben J Woodcroft ***@***.***>; Review requested ***@***.***> Subject: Re: [wwood/smafa] Add multithreading via rayon (PR #50) @AroneyS<https://github.com/AroneyS> requested your review on: #50<#50> Add multithreading via rayon. ― Reply to this email directly, view it on GitHub<#50 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAADX5CF564SQFWCXIGWIT32E5P4HAVCNFSM6AAAAABTMEBWBOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJVGYYDOOJVHE3DANI>. You are receiving this because your review was requested.Message ID: ***@***.***>

[AroneyS]

I tried running appraise using PATH="/home/aroneys/src/smafa/target/release:$PATH" singlem appraise
It took longer. 10h:36m for base, 10h:56m for dev version

Do you have a large-scale test for smafa directly?

[AroneyS]

With 600k sequences:
1 thread - 30 seconds
64 threads - 8 seconds