Allow parallel I/O for hydro and particle reading

[cphyc]

PR Summary

This allows parallel I/O for the RAMSES dataset. Reading a RAMSES dataset has three steps:

• read in the AMR structure
• read the hydro files
• read the particle files

The strategy I have adopted is to parallelize on the first one, such that each MPI task is now in charge of a few domains and only reads those (incl. hydro + particles).

[cphyc]

The motivation for getting rid of the cast to CHAR is so that we do not hit the limit imposed by MPI of 2**31 elements when communicating arrays as quickly.

[cphyc]

Isn't there a bug here? I think this should be

Suggested change

	self.comm.send(
	(arr.dtype.str, arr.shape, transport_dtype) + unit_metadata,
	dest=dest,
	tag=tag,
	)
	self.comm.Send([arr, mpi_type], dest=dest, tag=tag)
	self.comm.send(
	(arr.dtype.str, arr.shape, transport_dtype) + unit_metadata,
	dest=dest,
	tag=tag,
	)
	self.comm.Send([tmp, mpi_type], dest=dest, tag=tag)

[matthewturk]

I think it should be tmp, yeah, but since it has been working I think it's possible that mpi4py was fixing it implicitly.

[cphyc]

@yt-fido test this please

[cphyc]

After discussion with @matthewturk and @chrishavlin, we're proposing to implement this:

import numpy as np
import yt

ds = yt.load(...)
yt.enable_parallelism()
ad = ds.all_data()

def expensive_function(chunk):
    import time
    time.sleep(np.random.rand() * 3600)
    return 42

sto = {}
for chunk in ad.piter(storage=sto, reduction="min/max/sum/cat/cat_on_root"):
    sto.result["gas", "density"]     = chunk["gas", "density"]
    sto.result["gas", "temperature"] = chunk["gas", "temperature"]
    sto.result["expensive_stuff"]    = expensive_function(chunk)

if yt.is_root():
    mean_expensive_stuff = np.mean(sto["expensive_stuff"])
    plt.hist2d(sto["gas", "density"], sto["gas", "temperature"], bins=...)
    plt.title(f"{mean_expennsive_stuff=}")
    plt.savefig("...")

[cphyc]

Closing since #5218 is much better.