Trouble with non-standard parameters (Left-shift issue found by SonarQube)

[greyshepherd-star]

(1ULL << params->full_height)

always evaluates to 0 when params->full_height == 64 since ULL is a 64 bit integer.

e.g. xmss_core.c:122, ends up comparing idx with 0xFFFFFFFFFFFFFFFF.

    if (idx >= ((1ULL << params->full_height) - 1)) {
        // Delete secret key here. We only do this in memory, production code
        // has to make sure that this happens on disk.
        memset(sk, 0xFF, params->index_bytes);
        memset(sk + params->index_bytes, 0, (params->sk_bytes - params->index_bytes));
        if (idx > ((1ULL << params->full_height) - 1))
            return -2; // We already used all one-time keys
        if ((params->full_height == 64) && (idx == ((1ULL << params->full_height) - 1))) 
                return -2; // We already used all one-time keys
    }

Perhaps this is intentional, but if so it could be noted as such to help the reader.

[bwesterb]

This implementation only supports the standard parameter sets; none of which use H=64.

[greyshepherd-star]

That's seems a little odd since the comment above says "For the case of total tree height of 64 we do not use the last signature " and there is an explicit test for full_height == 64 in the fragment I included.