Skip to content

Incorrect results of correctedValue() if product of arguments are too big #1

@evanp

Description

@evanp

Weird bug: sub(30, 0.00004999999999833333) produces -0.0000499999999983333 for me.

sub( 9, 0.00004999999999833333) produces 8.999950000000002 which is more or less correct.

That long number has 20 digits to the right of the decimal place, which my NodeJS won't normally produce. Trimming a digit or two off the end of the number will return the right value.

I'm not sure what's going on; just thought I'd bring it up.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions