Skip to content

Incorrect results #3

@jaminturner

Description

@jaminturner

I think there is a problem with the two's complement implementation in this library. It is correct for positive numbers, but for negative numbers, it is incorrect. The math used to catch and convert negative numbers is:

if x_data & 0x80000 == 0x80000:
    x_data = ~x_data + 1

However, this just gives a negative version of x_data, it doesn't truly take it out of 2s complement form. For example, if the number received from the sensor is "1001 0110 0000 0010 1101", the correct decimal integer of that is -434131. However, the above code gives -614445, which is incorrect.

I think it could be corrected as follows:

if x_data & 0x80000 == 0x80000:
    x_data = x_data - 2**20

Am I wrong/confused on this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions