Estimation of instruction latency

[marshallward]

Instruction latency is currently hardcoded in the scalar tests (avx_add, etc). This can vary across architectures, e.g. 3 cycle on Sandy Bridge, 7 cycle on KNL. The latency is used to determine the loop unroll factor in these tests. Values which are too small can cause pipeline stalls and significantly reduce the peak performance.

Additionally, choosing a value that is too large can cause a slowdown. It is usually small, but is nonetheless suboptimal. Very large latency values can also consume registers and indirectly cause large slowdowns.

Agner Fog's tool will dynamically compute the latency of every instruction, albeit with a kernel module. It may be worth looking into this and seeing if we can use a similar method to estimate latency.

[marshallward]

A potential method for measuring instruction latency has been in the dev/ directory for quite a while now:

https://github.com/marshallward/optiflop/tree/main/dev/latency

Unfortunately, it "mostly" works. When it works, it's really quite impressive. It even captures the latency bounds of very slow instructions like sqrt. But it will occasionally just give a very wrong answer.

I think the idea will be for autoconf to run these test programs and compute the latency, then bang them into the source as define macros.

I feel like this is close, but still not ready to integrate into the codebase. (Which is a shame, but it is surely better than the current hardcoded macros.)