#586: Investigate erratic and unpredictable CI error regarding synthetic case (NEW)

[cwschilly]

Fixes #586

[ppebay]

Running 100 times the corresponding case (with only 8 iterations and 1 object per transfer):

for run in {1..100}; do python LBAF_app.py -c synthetic-acceptance.yaml; (cat ../../../output/imbalance.txt; echo) >> res; done

on macOS 14.3 with Python 3.9.19 yields:

awk '{ sum += $1 } END { print sum }' res
0

In other all 100 runs correctly converged to an imbalance of 0.0.

This seems to indicate rather a problem with the CI system itself, and specifically with Python 3.8
@cwschilly

[ppebay]

In keeping with the above comment, I created a Python 3.8 environment and performed the same test (100 runs):

for run in {1..100}; do python LBAF_app.py -c synthetic-acceptance.yaml; (cat ../../../output/imbalance.txt; echo) >> res; done

and obtained as well

awk '{ sum += $1 } END { print sum }' res
0

i.e., all 100 runs passed the test.

Specifically the version of Python is:

 python --version
Python 3.8.20

@cwschilly can we verify if this is the same version that runs in CI?

[cwschilly]

@ppebay The CI uses Python 3.8.18. I have been able to recreate the problem locally by running tox with Python 3.8.19, so I don't think it's just a problem with CI.

[cwschilly]

After changes to the ordering of ranks, Python 3.8 acceptance passed several times in a row. However, Python 3.11 failed: https://github.com/DARMA-tasking/LB-analysis-framework/actions/runs/13397121375/job/37418861102?pr=590