Skip to content

Conversation

@kdwink
Copy link

@kdwink kdwink commented Jun 13, 2014

Appears to show that serialized bytes of BigDecimal do not sort in natural order. Unless I am missing something?

Rest results:

Expected :[-0.5555555555555555555555555555555555, -0.5555555555555555, -0.555555555555555,
Actual :[-0.55, -0.5555, -0.555555, -0.55555555,

…ative values and positive and negative exponents. Appears to show that serialized bytes of BigDecimal do not sort in natural order.
@ndimiduk
Copy link
Owner

Definitely looks like you've found a bug. Any suggestion for a fix to the en/decoder?

I haven't been maintaining Orderly as much since OrderedBytes and DataType when into HBase. Any chance of transitioning over to that implementation?

@kdwink
Copy link
Author

kdwink commented Jun 16, 2014

I wasn't using orderly directly, rather something based on it. Attached test cases were an easy way to verify that the bug exists here too. In my case I added code to XOR the exponent and unscaled component of the bytes for negative numbers, then added an extra byte (always 0x01) to the end of the binary format so that relative order is stable for ascending and descending sorts. Otherwise N byte arrays compare wrong to N+1 byte arrays where the first N bytes are the same. Anyway, you may want to just close this pull request and log an issue so people are aware.... Cheers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants