unstable direct pre-training on Retinanet with different seeds

[belisarius0]

In retinenet's direct pre-train, I observed that loss becomes nan or inf during warmup when some seeds are specified for train.py.
Wrt default seed, No problem.
I hope to re-explore the hyper parameters to make the learning more stable.