Minimal example not fitting

[whilo]

I tried to get the minimal example in the README to work, and also looked through the experiments, but couldn't find something toyish and self-contained. I coded up this standard toy example, but I cannot get it to fit. Any ideas? https://gist.github.com/whilo/295440ff44c08b0a35bbdb1c18dd2409

[fdraxler]

Thanks for your interest! I ran the code with default hyperparameters and found the NLL unstable, suggesting beta was too low. Increasing beta to 100 and epochs to 1000 yielded a decent fit.

The surrogate loss doesn't indicate fit quality. I recommend monitoring the actual NLL from the encoder Jacobian to choose hyperparameters (this may become expensive in high dimensions). For final NLL reporting, be sure to also measure reconstruction quality, as low reconstruction quality can mean spurious likelihoods. Details and other training tips are in Appendix B of the Free-Form Flow paper (https://arxiv.org/pdf/2310.16624).

I modified your code to monitor NLL and used the above hyperparameters (plus a 2D latent distribution, required for exact_nll computation but shouldn't affect training). I also reduced the default number of hutchinson samples to 1, as we always use this. This didn't affect quality; increasing batch size reduces gradient variance more effectively than increasing hutchinson samples.

Here's the modified code: https://gist.github.com/fdraxler/4108667742dc1b7e4af1ba1739473c37