avx512 effect on performance

[ozgrakkurt]

Hello,

Thank you for creating this library. I ported it to zig here https://github.com/steelcake/zint/blob/main/src/fastlanes.zig

I have found disabling avx512 is much better on ryzen cpus for my code. I have tested this in an EPYC and a midrange desktop CPU.

On my implementation it speeds up delta and ffor encodings about two times.

I have also tested it on this repo by comparing

RUSTFLAGS='-C target-cpu=native' cargo bench --profile release

with

RUSTFLAGS='-C target-cpu=native -C target-feature=-avx512f' cargo bench --profile release

The only difference I could see was in rle decode. It goes from 15GB/s to 19GB/s

Disabling avx512 also leads to 2% slower performance on bitpacking on this repo which wasn't the case on my implementation.

Manually unrolling the transpose loop (like it is done here) seems to prevent the compiler from vectorizing it with avx512 so it also removes some of the disadvantage on my codebase but it generates a huge amount of assembly and feels like a hack. And disabling avx512 completely actually yields even better perf than just unrolling transpose loops on my case.

Are there other tricks similar to unrolling the tranpose loops that helps remove disadvantages of avx512? (I couldn't find any other difference between this implementation and mine)

Also curious why the code in this repo seems to be fine even with avx512 for the most part and it makes such a huge difference on mine. Maybe it is because rust/zig difference?

[gatesn]

Very neat! We also had a Zig version over here if you were interested, although it had different loop ordering that made the code size much larger as I think you also found. https://github.com/spiraldb/fastlanez/

The one thing I might note, that neither our Zig or Rust version support, is that I believe AVX512 is more sensitive to the performance of unaligned loads (at least vs Apple's M chips that I largely develop on). So it might be worth trying to align the inputs/outputs to 64 bytes?

If you were able to put this together in a GodBolt we could compare generated code and see where the difference is coming from?

[ozgrakkurt]

Very neat! We also had a Zig version over here if you were interested, although it had different loop ordering that made the code size much larger as I think you also found. https://github.com/spiraldb/fastlanez/

The one thing I might note, that neither our Zig or Rust version support, is that I believe AVX512 is more sensitive to the performance of unaligned loads (at least vs Apple's M chips that I largely develop on). So it might be worth trying to align the inputs/outputs to 64 bytes?

If you were able to put this together in a GodBolt we could compare generated code and see where the difference is coming from?

I learned some new language features from the Zig version! I figured the rust version is more maintained so I would port it instead of going more into the Zig implementation.

Thanks for the pointer. I'm trying to make data better aligned to see if it makes a difference.

I checked some assembly and it seems to be identical on both cases.

Another reason might be that the benchmarks I'm running are trying to read/write from different cache lines (not hit L1 cache) for every compress/decompress call but benchmarks on this repo use the same buffers for every bench iteration.

I'm also not benchmarking fastlanes functions (1024 length) only. I benchmark the top level function that handles the remainder data and makes repeated calls to underlying fastlanes functions.

So far I found that making the writes aligned (with an extra memcopy after) on cache line makes it 2x as fast in some cases. But making reads aligned (with extra memcopy) didn't make that much of a difference.

I'm trying to experiment with different variantions like making data at least aligned to T instead of being aligned to byte but not aligning it to cache line.

[ozgrakkurt]

Also found that avx512 has a perf disadvantage when handling 32bit integers and it ok when handling 64bit integers in my case

[gatesn]

This is super interesting, but I'm struggling a bit to follow all of your conclusions! Would you mind summarizing in a table or some sort?

[ozgrakkurt]

This is super interesting, but I'm struggling a bit to follow all of your conclusions! Would you mind summarizing in a table or some sort?

Avx512

In my implementation, disabling avx512 gives 50% performance boost on u32 delta encode/decode. But it doesn't make much of a difference on u64.

On this repo, disabling avx512 doesn't make a difference as far as can see but the benchmark uses u16 so that might be the reason. Disabling avx512 makes a decent difference (15GB/s -> 19GB/s) on RLE benchmark, which uses u32, so that might be an indicator.

When the transpose/untranspose loop in manually unrolled (before going into LLVM), disabling avx512 doesn't make that much of a difference for u32 delta. And it is unrolled with a macro on this repo. As a side note, manually unrolling that loop generates a huge amount of assembly.

I have found manually unrolling transpose/untranspose/delta/undelta(undelta_pack is used instead of this anyway) also gives perfect performance even when avx512 is enabled. I unroll these loops by a constant factor like 16. Just disabling avx512 and not doing any manual unrolling gives the best overall performance in my case though.

Disabling avx512 also improves performance in regular bitpack and ffor_pack for u32 in my implementation. So that might also be worth checking here.

Alignment

I tried adding a memcopy in order to align the input and output buffers. The only place this helps is the output buffer of unpacking function, and I found that it gives about 50% performance boost.

So I unpack into a 64 byte aligned buffer and then array assign (probably converted to memcopy) it. like this in zig:

const scratch_u: *[1024]U = @ptrCast(scratch);
offset += FL.dyn_bit_unpack(
    data_section[offset..],
    scratch_u,
    width,
);
out_offset[0..1024].* = scratch_u.*;

I realised this is the case because the signed versions of decompression functions were much faster than the unsigned versions. And the only difference was that they unpacked into an aligned buffer and then zigzag decoded that buffer into the real output(unaligned).

This alignment improvement is the same both when avx512 is enabled and disabled.

[ozgrakkurt]

@gatesn hope this is helpful! I couldn't get much more concrete numbers here sorry, I have to also move on from looking into this to keep my sanity 😆

I would recommend trying the downstream benchmarks like maybe on the vortex repo on an epyc/ryzen machine with and without avx512 if you got time, maybe it will be faster to disable it. I'll also try to do this if I can later