Skip to content

Expose JIT MPI communicator through to form#4107

Open
jhale wants to merge 5 commits intomainfrom
jhale/shared-mem-compile-comm
Open

Expose JIT MPI communicator through to form#4107
jhale wants to merge 5 commits intomainfrom
jhale/shared-mem-compile-comm

Conversation

@jhale
Copy link
Member

@jhale jhale commented Feb 26, 2026

Allows the compilation of forms on a communicator different to the mesh. The hypothesis is that this may be useful for setting up JIT cache on HPC runs across multiple nodes with shared memory-backed /tmp - this is still to test.

Example of use:

from mpi4py import MPI
from dolfinx.mesh import create_unit_square
from dolfinx.fem import functionspace, form
import ufl

mesh = create_unit_square(MPI.COMM_WORLD, 10, 10)
V = functionspace(mesh, ("CG", 1))
u = ufl.TrialFunction(V)
v = ufl.TestFunction(V)
a = ufl.inner(u, v)*ufl.dx

# See https://docs.open-mpi.org/en/v5.0.x/man-openmpi/man3/MPI_Comm_split_type.3.html#split-types
shared_mem_comm = MPI.COMM_WORLD.Split_type(MPI.COMM_TYPE_SHARED, key=MPI.COMM_WORLD.rank)
a_dolfinx = form(a, jit_comm=shared_mem_comm)

The default behaviour (use the communicator from the form's mesh) is retained when jit_comm=None, which is also the default argument.

@jhale jhale marked this pull request as ready for review February 26, 2026 14:54
Copy link
Member

@jorgensd jorgensd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should do it as a one-liner. Otherwise good

jhale and others added 3 commits February 26, 2026 16:20
Co-authored-by: Jørgen Schartum Dokken <dokken92@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants