Overflows

[agucova]

During testing of the correlations feature, I've noticed a bunch of distributions which casually produce overflows. Users wouldn't notice this except for infinities (np.inf) in their samples or distribution parameters.

I've fixed some of these (mostly those related to input parameters) alongside the distributions feature, but some that are still a problem:

• LognormalDistribution:
  • Outcome: Infinities in distribution parameters.
  • To replicate: sq.lognorm(1,1e+26)
  • Location:

    squigglepy/squigglepy/distributions.py

    Lines 773 to 775 in 9491e50

    	self.lognorm_sd = (
    	(np.exp(self.norm_sd**2) - 1) * np.exp(2 * self.norm_mean + self.norm_sd**2)
    	) ** 0.5

• LogTDistribution:
  • Outcome: Infinities in samples.
  • To replicate: np.any(sq.log_t_sample(1.0, 2.0, 1, 100_000, 90) == np.inf)
  • Location:

    squigglepy/squigglepy/samplers.py

    Lines 216 to 218 in 9491e50

    	np.exp(
    	normal_sample(mu, sigma, samples) / ((chi_square_sample(t, samples) / t) ** 0.5)
    	)

I think it would be hard to fix this, most of these errors come from limitations inherent of the floating point operations; but I would at least try to reraise the warning as an error, to prevent users from creating models which might be silently wrong.

[peterhurford]

Yeah my guess is that this occurs in underlying numpy right?

Hm yeah this seems pretty annoying. I imagine the user also wouldn't be excited for their programs to randomly crash with an error, though I imagine that is better than having unnoticed infinities. Resampling would also be complicated. I suppose rclip could be helpful here?

[agucova]

Hm yeah this seems pretty annoying. I imagine the user also wouldn't be excited for their programs to randomly crash with an error, though I imagine that is better than having unnoticed infinities. Resampling would also be complicated. I suppose rclip could be helpful here?

In some specific cases we can work around overflow and underflows by limiting calculations to a minimum or a maximum, but it's tough to estimate how they would impact the distributions.

For now I much prefer to raise an error unless there's an obvious fix (like bounding the parameters).

[peterhurford]

Yeah we could raise an error and recommend the user define their own bounds to make the error go away. I think that would be the best solution.