Convert to Criterion Benching

[michaelciraci]

Would you be opposed to me opening a PR to convert the benchmarks to be criterion based? I would like to investigate AVX512 butterfly performance, and this would make things easier.

In addition, the benchmarking harness currently in RustFFT cannot be run on stable Rust. As a result, the Rust community has largely moved to Criterion (Criterion is even suggested in the Rust Docs under benchmarking: cargo-bench).

[HEnquist]

This was discussed briefly a few years ago: #9

[michaelciraci]

As far as getting quicker results, we should be able to combine Criterion's measurement_time and warm_up_time to get that behavior. Furthermore, we could choose the accuracy vs. required time tradeoff.

It looks like getting the output on one line by default may be a bit harder, but I think the bench piped into a simple grep/rg should get that (but understand that built in would be preferred).

Just as a quick example:

$ cargo bench 2>&1 | rg time
64-f32                  time:   [39.640 ns 39.797 ns 39.966 ns]
64-f64                  time:   [64.728 ns 66.788 ns 69.337 ns]
128-f32                 time:   [80.495 ns 82.556 ns 85.065 ns]
128-f64                 time:   [144.05 ns 145.56 ns 147.51 ns]
256-f32                 time:   [197.34 ns 204.65 ns 212.89 ns]
256-f64                 time:   [357.37 ns 376.12 ns 397.74 ns]

[HEnquist]

I wrote that I would look into speeding up criterion, but I never got to it. Instead I spent some time playing with various settings to see if I could get the benches to give less noise. It was years ago so I don't remember exactly all I did, but I think disabling turbo was the most important single thing. I also made some simple script to run the benches several times to do some averaging. The results were good enough for what I needed.