Convergence / Stability info

[egull]

This is to keep track of our discussion Pavel/Sergei/Emanuel regarding possible instabilities.

Proposed course of action: I would just print warning if overlap is ill conditioned.
With suggestion to switch to damping

I would just print warning if overlap is ill conditioned.
With suggestion to switch to damping

On Tue, 2 Apr 2024 at 11:11, Emanuel Gull <egull@umich.edu> wrote:
Right.

Proposed course of action? Do nothing?

On Apr 2, 2024, at 11:00 AM, Sergei Iskakov <siskakov@umich.edu> wrote:

1. We print the condition number every iteration if the verbosity level is set to 2 and higher
2. During my tests I have not seen problem even if condition number is > 1e+10

On Tue, 2 Apr 2024 at 09:57, Emanuel Gull <egull@umich.edu> wrote:
How does the following course of action sound:

1. Print the condition number every iteration. Something like:
   DIIS Linear System Condition Number is: 1.e-7

2. If the condition number is below an arbitrary cutoff (proposal: 1e-7) additionally print the following warning:
   Warning: DIIS Linear System Condition Number below threshold of 1.e-7. Examine stability and consider switching to damping for future iterations.

Emanuel

On Apr 2, 2024, at 8:27 AM, Pavel Pokhilko <pokhilko@umich.edu> wrote:

In my LCIIS code I actually had a switch, but from a programmer's perspective it is not clear when exactly the instability occurs. It is not as simple as a bad condition number.

My code prints the condition number.

I accumulated experience by running more calculations. It looks like instabilities occur when changes in energy are around 1e-7 or smaller. This is when I can recommend switching to damping. This is not a very common type of calculation since such tight thresholds are rarely used for solids. I used 1e-7 to evaluate small energy differences in oxides.

Pavel

On Tue, Apr 2, 2024 at 12:20 PM Emanuel Gull <egull@umich.edu> wrote:
Right.

So, in practice, what should be done? I see the following options:

1. Do nothing and hope the users read Pavel’s paper
2. Print the condition number and a warning that one should consider switching to damping once the condition number reaches a threshold.
3. Actually switch to damping if the condition number is bad enough.

Sergei, Pavel, Do you have any preference or advice?

Emanuel

On Mar 30, 2024, at 12:13 PM, Pavel Pokhilko <pokhilko@umich.edu> wrote:

The round-off errors in CDIIS show up because the commutators become very small near convergence. You can see it if you read my paper about it. Several DIIS implementations that I saw actually switched to damping in this regime because the linear system gains a bad condition number.

Pavel

On Sat, Mar 30, 2024 at 12:09 PM Sergei Iskakov <siskakov@umich.edu> wrote:
Hi all,

I checked migrated DIIS/CDIIS against the old code.
I can confirm that they both work in single core and mpi cases. I noticed that the residual for CDIIS is much more susceptible to roundoff errors than conventional DIIS.

I will put small description on green web-site later today.

Best,
Sergei

[pavel-po]

I wrote Noise troubleshooting part of the tutorial fully explaining why the noise occurs and how to combat it in practice using BiVO3 system:
https://green-phys.org/docs/tutorials/tutorial_diis/#noise-troubleshooting