The compression function F (pq_func) is unnecessarily complicated

[joshuaehill]

You are calculating F(1/z) using standard scientific programming function calls (just directly calculating the function as specified using a standard function for the incomplete gamma function). In our setting, this isn't necessary; for positive integer k the sum form of the incomplete gamma function is just a polynomial scaled by some elements that cancel in our function (see https://dlmf.nist.gov/8.4#E8 for details). For k=2 (the only value that presently applies) this whole mess simplifies to an unbelievably simple polynomial: F(z) = 2 z^3 + 2 z^2 + z. You can just use this polynomial instead of the fancy math calls.

[dj-on-github]

Dang. I spent a long time working out high precision and large integer tolerant algorithms for computing incomplete gamma and binomial functions for my book. Hence my first instinct was to call them. The flawed APT tables came from overflow problems with the nist implementations. I’ll give that a go. Thank you.

…

On Tue, Sep 18, 2018 at 6:58 PM Joshua E. Hill ***@***.***> wrote: You are calculating F(1/z) using standard scientific programming function calls (just directly calculating the function as specified using a standard function for the incomplete gamma function). In our setting, this isn't necessary; for positive integer k the sum form of the incomplete gamma function is just a polynomial scaled by some elements that cancel in our function (see https://dlmf.nist.gov/8.4#E8 for details). For k=2 (the only value that presently applies) this whole mess simplifies to an unbelievably simple polynomial: F(z) = 2 z^3 + 2 z^2 + z. You can just use this polynomial instead of the fancy math calls. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHDBGUDt6BQyf0QSV_n-7oRrfXq5K2XFks5ucaTUgaJpZM4WvRY0> .