Prepare checking .cwasm files for reproducibility in CI

[chrysn]

Frankly, I have no clue how variable the output will be, esp. as we're using a nightly compiler.

I'd like to keep running those builds in CI for a bit as more PRs come in, just to get a feel of whether or not they are all already easily reproducible. Biggest factor will likely be the nightly version, but that's hard to test as there are not so many older Rust nightlies with which the now memory-reduced builds work in the first place.

(Also, no CI steps are active yet, thus draft).

[chrysn]

As both this action and my local builds should run on the same Rust nightly, and there are diffs, there is apparently some non-reproducibility. Where (and whether it's my setup or something that actually needs fixing in one of the tools) is to be found.

[anlavandier]

In my (limited) testing it seemed to be fairly reproducible given that we use the same nightly. The version of Wasmtime is also to be considered.

[chrysn]

Rebased onto main after the recent merges, so that things could work even after merging.

Ad reproducibility, the artifact generated by the CI run does differ in some files; diffoscope output looks like it's the ordering of the functions. As all involved systems are on latest nightlies and latest wasm-tools and wasmtime is pinned now in ./precompile_wasm.rs, I'm a bit clueless as to where else the variation could come from.

[anlavandier]

To be fair I didn't diff the files and just compared the binary size and saw that it stayed the same. Does the wc -c size and/or the size of the sections stay the same in your testing ?

[chrysn]

Size is the same. Diff says "yeah there's a difference". Sensible output is best obtained from diffoscope, which was built for Debian to show why a build is not reproducible, it automatically drills down as far as it can into any structures to show sensible deltas, while also showing differences that would not show at all in a plain "convert to readable format" diff.

[anlavandier]

Could this be OS-related. Also is there a way for me to test things ? I guess that the simplest way is to copy the actions workflow and open another PR ?

[chrysn]

If you arrive at the same binaries as I do, yes, a new PR is the easiest way to run tests on GitHub CI. One easier way to reproduce it may be to run in containers, but so far I don't have any data point as to whether things built in Docker would act like on my machine or like GitHub actions.

[anlavandier]

Which exact nightly did you use ? I'm seeing size diffs so that's probably because I didn't use the exact nightly you used

[chrysn]

I used 2025-11-02 I think, but right, by now that could be different, so we should probably pin that for reproducible builds (but update frequently).

[anlavandier]

After getting the nightly right, I don't get any diffs so that's promising

[anlavandier]

I rebased, pinned a nightly version and updated everything, I'll try to pick up the reproducibility issue from there