Make generation batch size tunable #5
Description
During scene generation, it's possible to run out of CUDA memory.
In the short term, we should intercept this error (RuntimeError: CUDA out of memory. ) and tell the user to lower the batch size.
This is something that in theory should be automatically configurable (knowing at render time what is the largest batch size that can fit in memory) - but that would require apriori knowledge about the memory usage of the model's forward pass. I'm thinking we can either
- provide "stats collection" routines that attach this kind of metadata to the model objects as presubmit hooks
- do a few "test" passes before doing the actual generation to figure out the relationship between batch size and memory usage (assuming linear).
Neither looks straight forward / worth it right now so punting on the automatic piece for now.