Add tool to upgrade collisions/closure in Hermes-3

[cmacmackin]

boutproject/hermes-3#399 changes the names of a number of components and also split some up. This PR provides a tool for automatically upgrading BOUT.inp files to handle these breaking changes. To do this, I slightly altered the API for the existing scripts that update input files, to make them more generic.

[cmacmackin]

Should I go ahead and merge this or is there anything else we're waiting for?

[ZedThree]

I guess we might just want to wait until the corresponding changes go into Hermes, in case any names change?

[cmacmackin]

The changes have been merged into Hermes, so I guess it makes sense to merge this now.

[mikekryjak]

The PR was merged prematurely. Let me finish my tests first

[mikekryjak]

Is there no way to just replace strings instead of clobbering all the formatting? It makes things a bit painful, especially for reactions:

 ################################################################################
 [reactions]
 diagnose = true
-type = (
-        d + e -> d+ + 2e,   # Deuterium ionisation
-        d+ + e -> d,        # Deuterium recombination
-        d + d+ -> d+ + d,   # Deuterium charge exchange
-        t + e -> t+ + 2e,   # Tritium ionisation
-        t+ + e -> t,        # Tritium recombination
-        t + t+ -> t+ + t,   # Tritium charge exchange
-        d + t+ -> d+ + t,   # D-T charge exchange
-        t + d+ -> t+ + d,   # T-D charge exchange
-        he + e -> he+ + 2e, # Helium ionisation
-        he+ + e -> he,      # Helium+ recombination
-       )
+type = ( d + e -> d+ + 2e, d+ + e -> d, d + d+ -> d+ + d, t + e -> t+ + 2e, t+ + e -> t, t + t+ -> t+ + t, d + t+ -> d+ + t, t + d+ -> t+ + d, he + e -> he+ + 2e, he+ + e -> he, )      # Deuterium ionisation  Deuterium recombination  Deuterium charge exchange  Tritium ionisation  Tritium recombination  Tritium charge exchange  D-T charge exchange  T-D charge exchange  Helium ionisation  Helium+ recombination

[ZedThree]

This builds on the more general purpose tool, which is based on parsing the input file. This means we can move inputs between sections, for instance. We're sacrificing the ability to preserve the formatting for correctness.

It might be possible to get it to only apply the changes to areas touched by fixes, but I'm not sure how much effort that would be.

[cmacmackin]

I think that would be quite challenging. The "normalisation" of the file (to use its own terminology) seems to be a result of having parsed the input file into Python classes, which are then written back out using standardised formatting.

[mikekryjak]

The script does a confusing thing. It adds braginskii_conduction to the top level, but doesn't rename ion_viscosity, thermal_force or collisions. Instead it redefines them as different components, e.g.:

[ion_viscosity]
type = braginskii_ion_viscosity

[collisions]
type = braginskii_collisions, braginskii_friction, braginskii_heat_exchange

I think this is confusing because at first glance the top level components look unchanged, making it more challenging to spot that component names have now changed, and resulting in an input file mixing new and old component names. The old component names are in the same place as before as well.

I would much prefer to make sure no old component name persists, and that the new components are all in the top level.

[cmacmackin]

The reason I took this approach was because the name of the component in the hermes:component variable doesn't necessarily correspond to the type of the component. Therefore, even if we change the names in the way you suggest we would still have to check each component section in case they explicitly declare one of the closure types. Basically, we'd still have to do the work that's happening now, plus some more. I figured it was simpler just to change the type names. This is particularly the case for the old collisions component, as that is now three components. Rather than worry about which configurations would need to go with each component, I just use one section for all of them.

The reason braginskii_conduction is handled differently is that there previously was no component for this. It was done within evolve_pressure/evolve_density, but those are handled on a per-species basis. We need braginskii_conduction to be a global component (at least until topological sorting is merged), so I couldn't just change/add component names to existing sections as I did for others.

[mikekryjak]

Running a simulation with an input file generated by the converter fails - the initialisation is in a loop like the below. I have tested it on @oparry-ukaea's recent test (https://github.com/boutproject/hermes-3/blob/1a2f9e303e38b956425d3e43d4893f736065aa2c/tests/integrated/1D-recycling-dthe/data/BOUT.inp) which includes all of the new components apart from electron viscosity.

	t+ conduction collisionality mode: 'multispecies' using t+_d+_coll t+_d_coll t+_d_cx t+_e_coll t+_he+_coll t+_he_coll t+_t+_coll t+_t_coll t+_t_cx 
	Option input:validate = 0 (default)
	Option input:error_on_unused_options = 1 (default)
	Option optionfile = BOUT.inp (default)
	Option datadir = ./dthene-after_closure (Command line)
	Option d+_density_error_integral = 0 (upstream_density_feedback)
Sim Time  |  RHS evals  | Wall Time |  Calc    Inv   Comm    I/O   SOLVER

0.000e+00          1       1.42e-01     2.1    0.0    0.1  106.9   -9.2
	Option species:d+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:d+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:d+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:d+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:d+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:d+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option species:t+:charge = 1 ()
	Option pardiv:type = cyclic (default)
	Option fft:fft_measurement_flag = estimate (default)
	Option species:d+:charge = 1 ()
	Option species:he+:charge = 1 ()
	Option species:t+:charge = 1 ()

I'm not sure what could have caused this...

[mikekryjak]

Ok, the print loop is actually a bug with thermal_force (boutproject/hermes-3#438).

@cmacmackin I can see that the way the upgrader deals with the new component names is more straightforward to implement, but I'm concerned that it will cause a lot of confusion for users, most of whom have never touched the component lists and don't have an in-depth understanding of the system. Would it be a lot of work to change the upgrader to rename the old components and keep everything top level?

I also noticed that the upgrader doesn't ask for which kind of upgrade to perform. Is the intent for the same upgrader to be used for any further changes and keep it called "hermes"?

[cmacmackin]

I also noticed that the upgrader doesn't ask for which kind of upgrade to perform. Is the intent for the same upgrader to be used for any further changes and keep it called "hermes"?

The way the BOUT++ upgraders handle this is to specify which version you are upgrading from/to. We could do that. It really comes down to how many layers of subcommands we want.

[cmacmackin]

I can see that the way the upgrader deals with the new component names is more straightforward to implement, but I'm concerned that it will cause a lot of confusion for users, most of whom have never touched the component lists and don't have an in-depth understanding of the system. Would it be a lot of work to change the upgrader to rename the old components and keep everything top level?

It's probably doable, it's just harder to cover every edge-case.

[cmacmackin]

@mikekryjak @ZedThree I've implemented Mike's requested changes. I guess this is probably ready to be merged now.

[mikekryjak]

Thanks Chris! Let me have a play early next week before it gets merged.

[cmacmackin]

@mikekryjak Have you had a chance to play with this at all yet? It would be good to get it merged, seeing as these changes to Hermes-3 are now in master.

[mikekryjak]

Sorry, still on leave. Will do next week

…

On Fri, 2 Jan 2026, 11:21 Chris MacMackin, ***@***.***> wrote: *cmacmackin* left a comment (boutproject/boutdata#137) <#137 (comment)> @mikekryjak <https://github.com/mikekryjak> Have you had a chance to play with this at all yet? It would be good to get it merged, seeing as these changes to Hermes-3 are now in master. — Reply to this email directly, view it on GitHub <#137 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO7DNNRWKAZ52TOD5W7FES34EZBAZAVCNFSM6AAAAACNVUCTFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMBUHE4TMMRVHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mikekryjak]

@cmacmackin thanks for the changes. I ran this on a test case and found that the script doesn't handle an existing block like this:

[collisions]
diagnose = true

Instead of changing the header to [braginskii_collisions], it appends this code to the end:

[braginskii_collisions]
diagnose = true

This leaves [collisions] still present and the code complains about it since we got rid of the component in the code.