Multithreading

[Sable-20]

Multithreading should be implemented. It can be noted that in the data.rs file there is a struct called Data with a field titled chunks

pub struct Data {
    size: AtomicUsize,
    human_readable_size: AtomicUsize,
    unit: Arc<RwLock<Units>>,
    chunk_count: AtomicUsize,
    chunks: Arc<RwLock<Vec<(AtomicUsize, AtomicUsize)>>>,
}

chunks field:

pub struct Data {
    ...
    chunks: Arc<RwLock<Vec<(AtomicUsize, AtomicUsize)>>>,
}

This is a vector of the full size of the wordlist in bytes chunked into a predetermined number of tuples.

Multithreading should be implemented so that each chunk can be downloaded separately and then merged so that downloads are quicker.

TODO:

• Determine best multithreading style (standard library, tokio, rayon, etc...)
• Implement this with the number of chunks based on the specified number of workers, or default number from config file if workers argument not passed to fetch