Undefined behavior in PACKEDANCESTRYMAP hash code

[chrchang]

The hasharr and hashit functions in src/admutils.c, which are used to compute the .snp / .ind hash values stored in the PACKEDANCESTRYMAP header, use signed integers that usually overflow in the middle of the computation. While unsigned-integer overflow is defined to "wrap around" in C, signed-integer overflow is "undefined behavior": compilers are allowed to assume signed-integer overflow never happens, and generate code that is correct in the no-overflow case but does weird things on overflow.

I'm guessing no existing compiler has actually done anything weird here, but it would be good to eliminate the possibility.

[bumblenick]

Chris: can you write to me directly: ***@***.***> Nick <

…

On Thu, Jun 19, 2025 at 12:24 AM Christopher Chang ***@***.***> wrote: *chrchang* created an issue (DReichLab/EIG#113) <#113> The hasharr and hashit functions in src/admutils.c, which are used to compute the .snp / .ind hash values stored in the PACKEDANCESTRYMAP header, use signed integers that usually overflow in the middle of the computation. While unsigned-integer overflow is defined to "wrap around" in C, signed-integer overflow is "undefined behavior": compilers are allowed to assume signed-integer overflow never happens, and generate code that is correct in the no-overflow case but does weird things on overflow. I'm guessing no existing compiler has actually done anything weird here, but it would be good to eliminate the possibility. — Reply to this email directly, view it on GitHub <#113>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEE77B3WOP6A6O7TPDC7TST3EI3PNAVCNFSM6AAAAAB7U2M452VHI2DSMVQWIX3LMV43ASLTON2WKOZTGE2TQOBXGM2DKNA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>