-
Notifications
You must be signed in to change notification settings - Fork 7
Open
Description
I think there is a problem with the two's complement implementation in this library. It is correct for positive numbers, but for negative numbers, it is incorrect. The math used to catch and convert negative numbers is:
if x_data & 0x80000 == 0x80000:
x_data = ~x_data + 1
However, this just gives a negative version of x_data, it doesn't truly take it out of 2s complement form. For example, if the number received from the sensor is "1001 0110 0000 0010 1101", the correct decimal integer of that is -434131. However, the above code gives -614445, which is incorrect.
I think it could be corrected as follows:
if x_data & 0x80000 == 0x80000:
x_data = x_data - 2**20
Am I wrong/confused on this?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels