Improvements to 01-basic-GAN

[hsteptoe]

Discussion and tracking of issues for improvement relating to the 01-basic-GAN model.

As of run 20220124T1701, running 01-basic-GAN/gan.py in 256x256 mode produces the following output:

[hsteptoe]

Initial tranche of improvements need to focus on:

• Check data normalisation transform. Is there a better method that would work with this source data (ie. only +ve values, check distribution etc.)
• Add code to transform GAN output back to data scale (ie. reverse the normalisation step) so we can compare with true data
• Code better diagnostics for comparison of GAN output vs true data
• General GAN architecture experimentation?

[hsteptoe]

General GAN architecture experimentation

• Consider visualising the feature extraction from layers. Can we take a more physical-based approach to designing the (top) layer(s) of the network? For example, given the 4.4km data, pick kernel/stride sizes with a physical basis. Maybe (flashtorch)[https://github.com/MisaOgura/flashtorch] could help here? https://towardsdatascience.com/feature-visualisation-in-pytorch-saliency-maps-a3f99d08f78a
• Remove pooling layers? Some literature suggests these are unnecessary, in favour of using larger strides to to reduce layer sizes: https://cs231n.github.io/convolutional-networks/, https://arxiv.org/abs/1412.6806v3 https://towardsdatascience.com/iclr-2015-striving-for-simplicity-the-all-convolutional-net-with-interactive-code-manual-b4976e206760
• Remove normalization layers? See also https://cs231n.github.io/convolutional-networks/ > Normalization Layer.
• Handy Conv calculator: https://madebyollin.github.io/convnet-calculator/