Available simulation results?

[dombrno]

Good evening,
I am using ALPSCORE/CT-HYB for the calculation of dynamic susceptibilities. This involves calculating G2, inverting the BSE equation, and performing the analytic continuation. The analytic continuation uses Maxent and thus needs some error information as input data. In order to feed a god estimate of the error to Maxent, I am considering a Jackknife resampling procedure on the output of the simulation. Hence, my questions:
a) is there anything better/easier than this available somewhere within the ALPS project?
b) if not, is the detailed simulation data available in the h5 output file?

Thanks a lot for your help.

[shinaoka]

Hi, I did not implement estimate of error bars simply because I did not see a need at that point.
There is no available error information in current h5 output file.

How do you propagate the errors of average sign in a Jackknife resampling?

[dombrno]

The idea would be to mimick a series of shorter QMC runs by splitting the full set of points of the simulation from a single run into bunches. We would then invert the BSE and obtain the dynamic susceptibility in imaginary bosonic frequency, based on each of these bunches. For this, what we would need is G2(l,l', omega_n) at each measurement point of the QMC simulation, if that is dumped anywhere?
Said differently, we are trying to avoid having to run a number of independent runs by analyzing the data of the long run we already have. but probably these data are not dumped, I would imagine this would be several Go large.

[dombrno]

In the old times, in the very first version of Alps, there was a function "evaluate" which actually had to be called once the QMC was finished, and did the all the averaging etc. Based on this architecture, it would have been possible to do the procedure mentioned above. But I understand that the new architecture probably does not give access to this fine-grained detail, which is not necessary, since the averaging is done at the end of the QMC run before results are dumped.

[shinaoka]

It may be possible with the accumulator or alea libraries in ALPSCore.
Markus Wallerberger may be the right person to answer this technical question.
Alex, Emanuel, could you assign him to this thread?

[shinaoka]

Alex, Emanuel, any idea?

[egull]

Yes. We have the FullBinning observable in the old ALPS. That one will keep a number of bins from the simulation (typically 128, I think) and fill them with the values. They are then written into the HDF5 file. A BS equation calculation can then take the data from the bins and post process them.
A word of warning though: this will require 128 times the storage for the HDF5 file (and in memory), which is substantial for vertex functions.
Hiroshi, to enable this you would have to find the accumulator and change it from mean or nobinning to fullbinning. @dombrno would then have to rerun the calculation.
With the new ALPS we have the same capabilities, just the rewrite of the ALEA has the binning procedure optimized a bit. A paper is in preparation.

[dombrno]

I understand, that makes sense: I remember that in the course of adapting the old Alps code (segment) for my needs (expand it to two orbitals, and use non diagonal hybridization function), I replaced the proposed accumulators with others which did not do any binning, and enjoyed a much more compact output file (I did not check memory consumption, and I was only calculating single-particle GF).

[egull]

So... how can we help? Should we get you a version that can do binning so you can try?

[dombrno]

I think that before anything is done, I should assess the memory and storage which are needed at the moment, and consider how much more the hardware is able to handle on my side.

[shinaoka]

Thank you, Emanuel.

@dombrno
I want to ask you a real need for your BS equation stuff.
What if you just run the CT-HYB solver several times with different random seeds?
This may also give some estimates of error bars.

[dombrno]

Yes, that is an option.

Maybe we can keep this feature as a "nice to have" option, but I would give it the lowest priority.

[dombrno]

For what it's worth, with the current code, the h5 file is 100 Mo large, while, and the calculation consumes 5Go RAM. The available RAM is 128Go on my hardware, so I could probably use up to 20 bins, if this option were ever implemented.

If I do have a real strong need for it, I will ask for guidance as to which type of accumulator is best to use in your opinion, and implement it on my side, but for the time being I thank you for your answer and your help, which perfectly answers my initial question.