Decreased preformance on version 0.7.0 and newer

[p-314]

I was running the big integer benchmark and the performance seems the decrease on version 0.7.0 compared to 0.6.1. Calculating e to a million places takes about 7000 ms on version 0.7.0 and newer and only 800 ms on 0.6.1. These timings are also consistent with the other benchmarks available in that crate.

I'm compiling on stable-x86_64-unknown-linux-gnu but also tested it on Windows.

[mhogrefe]

That's concerning. This is what I get using 0.7.0:

mikhailhogrefe@Mikhails-MacBook-Pro-2 bigint-benchmark-rs % cargo run --release -- --task e --lib rug --lib malachite -n 1000000 benchmark
    Finished `release` profile [optimized] target(s) in 0.09s
     Running `target/release/bigint-benchmark --task e --lib rug --lib malachite -n 1000000 benchmark`
Benchmarking:
    rug
        attempt 1: 112 ms (128 iterations)
        attempt 2: 112 ms (128 iterations)
        attempt 3: 112 ms (128 iterations)
        attempt 4: 113 ms (128 iterations)
        attempt 5: 114 ms (128 iterations)
    malachite
        attempt 1: 143 ms (128 iterations)
        attempt 2: 143 ms (128 iterations)
        attempt 3: 143 ms (128 iterations)
        attempt 4: 144 ms (128 iterations)
        attempt 5: 143 ms (128 iterations)
Results:
    rug        112 ms
    malachite  143 ms

Are you sure you're running in release mode? And could you possibly be using no_std? Malachite 0.7.0 does not enable no_std by default, though.

[mhogrefe]

When I run without --release I get about 8600 ms per attempt. Hopefully that's what you were doing too...

[p-314]

I'm definitely using release mode (debug takes ~60s). Also no #[no_std]. I've also benchmarked using criterion and the time command and they all agree. Running on stable-x86_64-pc-windows-mscv is also consistent, although it is "only" ~2.5x slower compared to version 0.6.1.

I copied some of the code from bigint-benchmark into here.

[mhogrefe]

Running your benchmark on my Mac is also fast:

mikhailhogrefe@Mikhails-MacBook-Pro-2 malachite_bench % time cargo run --release
    Finished `release` profile [optimized] target(s) in 0.00s
     Running `target/release/malachite_bench`
2.71828182845904523536028747135266249775724709369995957496696762772407663035354759457138217852516642
cargo run --release  0.21s user 0.03s system 60% cpu 0.399 total

I don't have a Windows machine to replicate this on, so I'm setting one up through Google Compute Engine.

[mhogrefe]

I experimented with a build on Windows. It looks like setting the wide dependency to the latest version, 1.1.1, helps with performance. I also want to try changing the global allocator, but I don't have the proper build tools to test that. Tomorrow I'm going to try on a Linux machine, which should be easier to work with.

[mhogrefe]

Alright, I did some profiling on Linux, and it's pretty clear that the fma and rint instructions are much slower there than on my Mac. Compare these flamegraphs, both zoomed into sd_fft_basecase_8_0:

Mac:

Linux:

[mhogrefe]

Correction: On my Mac many of those calls don't happen at all, because a faster implementation is selected here, but not on Linux. Basically, on some platforms mul_add isn't sufficiently accurate, so we have to use the slower implementation.

But I don't understand why the slower implementation is selected for stable-x86_64-unknown-linux-gnu. I copied my config from here; I would expect my test Linux env to pass, because its target arch is x86_64 and it seems to support fma.

Hopefully I'll be able to revise the feature configs and enable the faster implementation on this Linux target.

[mhogrefe]

@p-314 Could you try running your benchmarks with RUSTFLAGS="-C target-feature=+fma"? If that improves performance I'll set it automatically within Malachite. In the meantime I'm running all the malachite-nz tests with that feature on Linux to confirm that nothing breaks.

[p-314]

RUSTFLAGS="-C target-feature=+fma" cargo build -r does not compile on my Linux machine.

error: failed to run custom build command for `libm v0.2.15`

Caused by:
  process didn't exit successfully: `/home/pascal/rust/malachite_bench/target/release/build/libm-6a9960e00e71d160/build-script-build` (signal: 4, SIGILL: illegal instruction)

(Just a Chromebook with an Intel Pentium that doesn't support FMA. I wonder how rug handles that.)

But it does work on Windows (Ryzen 5 7600X). It is about 45% faster compared to version 0.6.1.

[mhogrefe]

Good that it works on Windows, at least. rug's multiplication is based on GMP, and Malachite's is based on FLINT; only the FLINT implementation uses fma.

Does RUSTFLAGS='-C target-cpu=native' work on your Linux machine?

If not, I'm not sure what else to try. Bringing back the 0.6.* implementation isn't really an option; my initial motivation for rewriting multiplication was that I found a bug in my old implementation that I was unable to debug. Usually I try running the same computation on a local GMP build, but this time it kept segfaulting.

In my next release I'll modify the config so that you'll get good performance from your Windows build without needing to pass any flags, and upgrade wide to 1.1.1.