Skip to content

Combine compute_perturbed_quantities_and_interpolation further #896

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
muellch wants to merge 136 commits into
base: main
Choose a base branch
from
Open

Combine compute_perturbed_quantities_and_interpolation further #896

muellch wants to merge 136 commits into from
+56 −326

Conversation

@muellch
Copy link
Contributor

@muellch muellch commented Oct 8, 2025

No description provided.

DropD and others added 30 commits September 4, 2025 14:42
@DropD
add first batch of program tests to continuous benchmarking
a460d50
@DropD @havogt
fix static params for apply_diffusion_to_vn
84abc04 
Co-authored-by: Hannes Vogt <hannes@havogt.de>
@DropD @havogt
improve static params for apply_diffusion_to_w_... test
50e8365 
Co-authored-by: Hannes Vogt <hannes@havogt.de>
@DropD
Merge branch 'main' into expand-continuous-benchmarking
ad8d449
@DropD
add static params and remove indirectly used prog from benchmarking
aacd35b
@DropD
update dockerfile to trigger a rebuild with newer bencher CLI
d042c98
@DropD
add compute_derived_horizontal_winds_... to benchmarking
ba94c10
@DropD
Merge branch 'main' into expand-continuous-benchmarking
4571a34
@DropD
temporarily remove failing parametrization case.
3dcdd34
@DropD
add compute_advection_in_vertical... program
367ba1f
@DropD
add benchmarking for advection_in_horizontal... program
c4b33c3
@DropD
add benchmarking for compute_perturbed... program
bdc3296
@DropD
add benchmarking for 'compute_hydrostatic...' program
5836fbf
@DropD
fix precommit messages
7d041d5
@OngChia @DropD
fix a bug in grid-savepoint reading during driver initialization (#866)
a30e5da 
Fix bug in the input arguments of
`IconSerialDataProvider.from_grid_savepoint` in
`initialization_utils.py` of the driver package after this PR
#829 is merged.
@havogt @DropD
Fix cupy cfl reduction performance by raveling the array (#872)
ca2f4f2 
On the mch-ch1_medium experiment
- this is 4% faster on the full timestep compared to the version without
`ravel`;
- within fluctuations there is no difference between no reduction and
this version.
@philip-paul-mueller @edopao @DropD
Update to GT4Py v1.0.9 (dace fix) (#840)
36f5583 
Co-authored-by: Edoardo Paone <edoardo.paone@cscs.ch>
@halungge @DropD
delete outdated top level README.md from tools (#870)
510dfae 
delete top level README.md from tools
@dependabot @DropD
Bump actions/setup-python from 5 to 6 (#863)
3824293 
Bumps [actions/setup-python](https://github.com/actions/setup-python)
from 5 to 6.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/setup-python/releases">actions/setup-python's
releases</a>.</em></p>
<blockquote>
<h2>v6.0.0</h2>
<h2>What's Changed</h2>
<h3>Breaking Changes</h3>
<ul>
<li>Upgrade to node 24 by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/1164">actions/setup-python#1164</a></li>
</ul>
<p>Make sure your runner is on version v2.327.1 or later to ensure
compatibility with this release. <a
href="https://github.com/actions/runner/releases/tag/v2.327.1">See
Release Notes</a></p>
<h3>Enhancements:</h3>
<ul>
<li>Add support for <code>pip-version</code> by <a
href="https://github.com/priyagupta108"><code>@​priyagupta108</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1129">actions/setup-python#1129</a></li>
<li>Enhance reading from .python-version by <a
href="https://github.com/krystof-k"><code>@​krystof-k</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/787">actions/setup-python#787</a></li>
<li>Add version parsing from Pipfile by <a
href="https://github.com/aradkdj"><code>@​aradkdj</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/1067">actions/setup-python#1067</a></li>
</ul>
<h3>Bug fixes:</h3>
<ul>
<li>Clarify pythonLocation behaviour for PyPy and GraalPy in environment
variables by <a
href="https://github.com/aparnajyothi-y"><code>@​aparnajyothi-y</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1183">actions/setup-python#1183</a></li>
<li>Change missing cache directory error to warning by <a
href="https://github.com/aparnajyothi-y"><code>@​aparnajyothi-y</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1182">actions/setup-python#1182</a></li>
<li>Add Architecture-Specific PATH Management for Python with --user
Flag on Windows by <a
href="https://github.com/aparnajyothi-y"><code>@​aparnajyothi-y</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1122">actions/setup-python#1122</a></li>
<li>Include python version in PyPy python-version output by <a
href="https://github.com/cdce8p"><code>@​cdce8p</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/1110">actions/setup-python#1110</a></li>
<li>Update docs: clarification on pip authentication with setup-python
by <a
href="https://github.com/priya-kinthali"><code>@​priya-kinthali</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1156">actions/setup-python#1156</a></li>
</ul>
<h3>Dependency updates:</h3>
<ul>
<li>Upgrade idna from 2.9 to 3.7 in /<strong>tests</strong>/data by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/setup-python/pull/843">actions/setup-python#843</a></li>
<li>Upgrade form-data to fix critical vulnerabilities <a
href="https://redirect.github.com/actions/setup-python/issues/182">#182</a>
&amp; <a
href="https://redirect.github.com/actions/setup-python/issues/183">#183</a>
by <a
href="https://github.com/aparnajyothi-y"><code>@​aparnajyothi-y</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1163">actions/setup-python#1163</a></li>
<li>Upgrade setuptools to 78.1.1 to fix path traversal vulnerability in
PackageIndex.download by <a
href="https://github.com/aparnajyothi-y"><code>@​aparnajyothi-y</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1165">actions/setup-python#1165</a></li>
<li>Upgrade actions/checkout from 4 to 5 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/setup-python/pull/1181">actions/setup-python#1181</a></li>
<li>Upgrade <code>@​actions/tool-cache</code> from 2.0.1 to 2.0.2 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/setup-python/pull/1095">actions/setup-python#1095</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/krystof-k"><code>@​krystof-k</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/setup-python/pull/787">actions/setup-python#787</a></li>
<li><a href="https://github.com/cdce8p"><code>@​cdce8p</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-python/pull/1110">actions/setup-python#1110</a></li>
<li><a href="https://github.com/aradkdj"><code>@​aradkdj</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-python/pull/1067">actions/setup-python#1067</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-python/compare/v5...v6.0.0">https://github.com/actions/setup-python/compare/v5...v6.0.0</a></p>
<h2>v5.6.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Workflow updates related to Ubuntu 20.04 by <a
href="https://github.com/aparnajyothi-y"><code>@​aparnajyothi-y</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1065">actions/setup-python#1065</a></li>
<li>Fix for Candidate Not Iterable Error by <a
href="https://github.com/aparnajyothi-y"><code>@​aparnajyothi-y</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1082">actions/setup-python#1082</a></li>
<li>Upgrade semver and <code>@​types/semver</code> by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/1091">actions/setup-python#1091</a></li>
<li>Upgrade prettier from 2.8.8 to 3.5.3 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/1046">actions/setup-python#1046</a></li>
<li>Upgrade ts-jest from 29.1.2 to 29.3.2 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/1081">actions/setup-python#1081</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-python/compare/v5...v5.6.0">https://github.com/actions/setup-python/compare/v5...v5.6.0</a></p>
<h2>v5.5.0</h2>
<h2>What's Changed</h2>
<h3>Enhancements:</h3>
<ul>
<li>Support free threaded Python versions like '3.13t' by <a
href="https://github.com/colesbury"><code>@​colesbury</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/973">actions/setup-python#973</a></li>
<li>Enhance Workflows: Include ubuntu-arm runners, Add e2e Testing for
free threaded and Upgrade <code>@​action/cache</code> from 4.0.0 to
4.0.3 by <a
href="https://github.com/priya-kinthali"><code>@​priya-kinthali</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1056">actions/setup-python#1056</a></li>
<li>Add support for .tool-versions file in setup-python by <a
href="https://github.com/mahabaleshwars"><code>@​mahabaleshwars</code></a>
in <a
href="https://redirect.github.com/actions/setup-python/pull/1043">actions/setup-python#1043</a></li>
</ul>
<h3>Bug fixes:</h3>
<ul>
<li>Fix architecture for pypy on Linux ARM64 by <a
href="https://github.com/mayeut"><code>@​mayeut</code></a> in <a
href="https://redirect.github.com/actions/setup-python/pull/1011">actions/setup-python#1011</a>
This update maps arm64 to aarch64 for Linux ARM64 PyPy
installations.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/setup-python/commit/e797f83bcb11b83ae66e0230d6156d7c80228e7c"><code>e797f83</code></a>
Upgrade to node 24 (<a
href="https://redirect.github.com/actions/setup-python/issues/1164">#1164</a>)</li>
<li><a
href="https://github.com/actions/setup-python/commit/3d1e2d2ca0a067f27da6fec484fce7f5256def85"><code>3d1e2d2</code></a>
Revert &quot;Enhance cache-dependency-path handling to support files
outside the w...</li>
<li><a
href="https://github.com/actions/setup-python/commit/65b071217a8539818fdb8b54561bcbae40380a54"><code>65b0712</code></a>
Clarify pythonLocation behavior for PyPy and GraalPy in environment
variables...</li>
<li><a
href="https://github.com/actions/setup-python/commit/5b668cf7652160527499ee14ceaff4be9306cb88"><code>5b668cf</code></a>
Bump actions/checkout from 4 to 5 (<a
href="https://redirect.github.com/actions/setup-python/issues/1181">#1181</a>)</li>
<li><a
href="https://github.com/actions/setup-python/commit/f62a0e252fe7114e86949abfa6e1e89f85bb38c2"><code>f62a0e2</code></a>
Change missing cache directory error to warning (<a
href="https://redirect.github.com/actions/setup-python/issues/1182">#1182</a>)</li>
<li><a
href="https://github.com/actions/setup-python/commit/9322b3ca74000aeb2c01eb777b646334015ddd72"><code>9322b3c</code></a>
Upgrade setuptools to 78.1.1 to fix path traversal vulnerability in
PackageIn...</li>
<li><a
href="https://github.com/actions/setup-python/commit/fbeb884f69f0ac1c0257302f62aa524c2824b649"><code>fbeb884</code></a>
Bump form-data to fix critical vulnerabilities <a
href="https://redirect.github.com/actions/setup-python/issues/182">#182</a>
&amp; <a
href="https://redirect.github.com/actions/setup-python/issues/183">#183</a>
(<a
href="https://redirect.github.com/actions/setup-python/issues/1163">#1163</a>)</li>
<li><a
href="https://github.com/actions/setup-python/commit/03bb6152f4f691b9d64579a1bd791904a083c452"><code>03bb615</code></a>
Bump idna from 2.9 to 3.7 in /<strong>tests</strong>/data (<a
href="https://redirect.github.com/actions/setup-python/issues/843">#843</a>)</li>
<li><a
href="https://github.com/actions/setup-python/commit/36da51d563b70a972897150555bb025096d65565"><code>36da51d</code></a>
Add version parsing from Pipfile (<a
href="https://redirect.github.com/actions/setup-python/issues/1067">#1067</a>)</li>
<li><a
href="https://github.com/actions/setup-python/commit/3c6f142cc0036d53007e92fa1e327564a4cfb7aa"><code>3c6f142</code></a>
update documentation (<a
href="https://redirect.github.com/actions/setup-python/issues/1156">#1156</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/setup-python/compare/v5...v6">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/setup-python&package-manager=github_actions&previous-version=5&new-version=6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@iomaganaris @OngChia @DropD
Clean up 41_to_60 for better performance (#784)
1c899eb 
- Removed `vertical_mass_flux_at_cells_on_half_levels`,
`tridiagonal_alpha_coeff_at_cells_on_half_levels`,
`tridiagonal_beta_coeff_at_cells_on_model_levels`, `exner_explicit_term`
and `rho_explicit_term` from intermediate fields passed around
- Combined
`_vertically_implicit_solver_at_corrector/predictor_step_before/after_solving_w`
- Improved initialization of scans (level 0 and 80) (together with
@philip-paul-mueller )
- Removed any unnecessary initializations related to them (i.e.
`next_w`)
  - Handle related unit tests
- Make sure that the values for the `n_lev` vertical level are passed
from `_set_surface_boundary_condtion_for_computation_of_w` to
`_vertically_implicit_solver_at_corrector/predictor_step_before_solving_w`
properly
  - Improves DaCe handling of temporaries

---------

Co-authored-by: "Ong Chia Rui" <chiarui.ong@c2sm.ethz.ch>
@edopao @DropD
Fix memory allocator for zero fields in velocity advection tests (#874)
937128b 
Fix an issue in velocity advection tests, that caused the dace programs
to fail: a cupy array was expected as argument, a numpy array was
passed.
@edopao @DropD
Add dace backend to default CI pipeline (#822)
c1bfd1c 
This PR enables CI tests on the dace backend.
- In the Github Actions, we run the stencil tests on the CPU backend.
- In the CSCS CI, we run the tests with serialized data on the GPU
backend, only for diffusion and dycore in order to save compute
resources.
@DropD
Merge branch 'main' into expand-continuous-benchmarking
02d031f
@DropD
add benchmarking for 'vertically_implicit_solver...' program
bddb09d
@DropD
avoid dace backend corner cases in stencil tests
48c9b08
@DropD
Merge branch 'main' into expand-continuous-benchmarking
4ea9499
@DropD
add benchmarking for 'interpolate_rho_theta...' program
70e1720
@DropD
fix typing for static variants in stencil tests
1479e80
@DropD
add benchmarking for 'apply_divergence_damping...' program
cf730e8
@DropD
add benchmarking for 'vertically_implicit...corrector...' program
47d647a
@edopao
Copy link
Contributor

edopao commented Dec 9, 2025

I have merged main, which includes the new version of gt4py with the fix for the dace backend.

@edopao
Copy link
Contributor

edopao commented Dec 9, 2025

cscs-ci run default

@nfarabullini
Copy link
Contributor

nfarabullini commented Dec 15, 2025

cscs-ci run default

@nfarabullini nfarabullini requested a review from edopao December 15, 2025 12:53
@edopao
Copy link
Contributor

edopao commented Dec 15, 2025

cscs-ci run dace

@edopao
Copy link
Contributor

edopao commented Dec 15, 2025

DaCe tests pass in ICON4Py CI. I see errors (segfaults) only in ICON-blueline runs, with the dace backend, while it still works with the gtfn backend. Based on the discussion on Slack, I suspect that issue of these errors is in GT4Py-DaCe. There is no objection from my side about merging this PR, because the DaCe issue needs to be fixed in GT4Py anyway.

@havogt
Copy link
Contributor

havogt commented Dec 15, 2025

I don't think we should merge if it doesn't work in blueline. The main goal of this refactoring is to get better performance in the blueline.

@muellch
Copy link
Contributor Author

muellch commented Dec 16, 2025

I can try to figure out where the out of bound access happens by either staring at the code or running blue line with cuda-memcheck.

@havogt havogt mentioned this pull request Dec 18, 2025
Closed
havogt
havogt requested changes Dec 18, 2025
View reviewed changes
Copy link
Contributor

@havogt havogt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remind me of what was the goal of this PR? Did we want to combine more such that intermediate values on half or full levels disappear? Maybe that's still to do?

@nfarabullini
Copy link
Contributor

nfarabullini commented Dec 19, 2025

cscs-ci run default

@nfarabullini
Copy link
Contributor

nfarabullini commented Dec 22, 2025

cscs-ci run default

@edopao
Copy link
Contributor

edopao commented Dec 23, 2025

cscs-ci run default

@edopao
Copy link
Contributor

edopao commented Dec 23, 2025

The last commit removed a write-write pattern in compute_perturbed_quantities_and_interpolation. The optimized SDFG now contains a single access node to exner_at_cells_on_half_levels:
Screenshot 2025-12-23 at 11 47 00

This change has two advantages:

  1. The new SDFG leads to better performance for compute_perturbed_quantities_and_interpolation, see the ~5% additional speedup in the attached plot (the blue bar is always the OpenACC baseline, then 2 bars for icon4py main and the last 2 bars for this PR).
  2. By removing the write-write pattern, we can workaround an issue of DaCe state fusion.
bench_blueline_stencil_compute

@github-actions
Copy link

github-actions bot commented Jan 5, 2026

Mandatory Tests

Please make sure you run these tests via comment before you merge!

  • cscs-ci run default

Optional Tests

To run benchmarks you can use:

  • cscs-ci run benchmark-bencher

To run tests and benchmarks with the DaCe backend you can use:

  • cscs-ci run dace

To run test levels ignored by the default test suite (mostly simple datatest for static fields computations) you can use:

  • cscs-ci run extra

For more detailed information please look at CI in the EXCLAIM universe.

@nfarabullini
Copy link
Contributor

nfarabullini commented Jan 5, 2026

cscs-ci run default

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nfarabullini nfarabullini nfarabullini left review comments

@havogt havogt havogt requested changes

@edopao edopao Awaiting requested review from edopao

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.