CDKLM nonsymmetric

[babrodtk]

CDKLM is no longer symmetric. In commit 35ad712 it appears to work well and give symmetric results, but in current master, c93d524, it is nonsymmetric.

Uncover by simulating a symmetric case, and plt.imshow(eta - eta[::-1, ::-1]) The more timesteps it runs it appears to become more unsymmetric.

Uncovered by running the NumericalOrder.ipynb notebook

[havahol]

The sinner seems to be the --use_fast_math compiler flag.
By removing this flag, the CDKLM scheme produce results with the same symmetric quality as in the commit 35ad712.

However, removing the fast math flag will result in a slow down of factor 2, which should be avoided.
According to the nvcc docs the flag "[m]ake[s] use of fast math library. --use_fast_math implies --ftz=true --prec-div=false --prec-sqrt=false --fmad=true." By some experimentation, it turns out that --ftz=true --prec-div=false --prec-sqrt=false --fmad=false gives the same (or even slightly better) symmetric properties as without fast math, with only a 2% performance loss.

Results from a symmetric test case eta - eta[::-1, ::-1] with fast math:

Results from a symmetric test case eta - eta[::-1, ::-1] without fast math:

Results from a symmetric test case eta - eta[::-1, ::-1] with optimal configuration:

Note that fmad is true on default, which might have caused the asymmetry in the case without fast math.

At the time of writing: Tested on one device only.