Fortran runtime error: Allocatable argument 'domains' is not allocated

[arthurronner]

Hi,

I have been trying to get mizuRoute to run for a simple case (the Rhône river basin for a runtime of years) on the ALICE HPC of Leiden University, but I am getting stuck at the listed error.

As my input, I'm using the MERIT rivers and basins topology (which contains 2114 river segments all routing to 1 outlet), and input runoff data from watergap, formatted to netCDF using easymore. I've used the same ID's for both hru and seg, as MERIT has a 1-1 relation between basins and river segments.
The netCDF files are formatted as follows:

------- RUNOFF DATA -------
Dimensions:    (ID: 2114, time: 60)
Coordinates:
  * time       (time) object 480B 2015-01-01 00:00:00 ... 2019-12-01 00:00:00
  * ID         (ID) float64 17kB 2.1e+07 2.1e+07 2.1e+07 ... 2.1e+07 2.101e+07
Data variables:
    latitude   (ID) float64 17kB ...
    longitude  (ID) float64 17kB ...
    qtot       (time, ID) float64 1MB ...
Attributes:
    Conventions:    CF-1.6
    License:        GPLv3+; Originla license: No license from original file i...
    History:        Created Thu Sep 18 09:55:37 2025by EASYMORE nc_remapper; ...
    easymore_hash:  8abcf88023ff16ff121b55ed4668b9cd8871660e0f7ba8fb700e71fc5...
    Source:         Remapped by EASYMORE nc_remapper from original file: C:\U...
<xarray.Dataset> Size: 118kB

------- RIVER TOPOLOGY -------
Dimensions:    (seg: 2114, hru: 2114)
Coordinates:
  * seg        (seg) int64 17kB 0 1 2 3 4 5 6 ... 2108 2109 2110 2111 2112 2113
  * hru        (hru) int64 17kB 0 1 2 3 4 5 6 ... 2108 2109 2110 2111 2112 2113
Data variables:
    segId      (seg) float64 17kB ...
    HRUid      (hru) float64 17kB ...
    length     (seg) float64 17kB ...
    slope      (seg) float64 17kB ...
    downSegId  (seg) float64 17kB ...
    hruSegId   (hru) float64 17kB ...
    area       (hru) float64 17kB ...

Below is the full output of mizuRoute. I've installed mizuRoute 3.0.0 using the given instructions on the wiki.
Do you have any experience with this error? How could I go about solving it?

Best,
Arthur

At line 754 of file /home/ronnerac/mizuRoute/mizuRoute_code/route/build/src/domain_decomposition.f90
Fortran runtime error: Allocatable argument 'domains' is not allocated

Error termination. Backtrace:
#0  0x53eb66 in assign_node
	at /home/ronnerac/mizuRoute/mizuRoute_code/route/build/src/domain_decomposition.f90:754
#1  0x55da50 in __domain_decomposition_MOD_mpi_domain_decomposition
	at /home/ronnerac/mizuRoute/mizuRoute_code/route/build/src/domain_decomposition.f90:80
#2  0x746289 in __init_model_data_MOD_init_ntopo_data
	at /home/ronnerac/mizuRoute/mizuRoute_code/route/build/src/init_model_data.f90:276
#3  0x75cc1d in __model_setup_MOD_init_data
	at /home/ronnerac/mizuRoute/mizuRoute_code/route/build/src/standalone/model_setup.f90:95
#4  0x75d890 in route_runoff
	at /home/ronnerac/mizuRoute/mizuRoute_code/route/build/src/standalone/route_runoff.f90:5
#5  0x75db60 in main
	at /home/ronnerac/mizuRoute/mizuRoute_code/route/build/src/standalone/route_runoff.f90:11
 (t_initf) Read in prof_inparm namelist from: dummy_empty_namelist
 (t_initf) Using profile_disable= F  profile_timer=           4
 (t_initf)  profile_depth_limit=       99999  profile_detail_limit=           1
 (t_initf)  profile_barrier= F  profile_outpe_num=           0
 (t_initf)  profile_outpe_stride=           1  profile_single_file= F
 (t_initf)  profile_global_stats= T  profile_papi_enable= F

---- read control file --- 
 <case_name> --> RHONE_TEST
 <sim_start> --> 2015-01-01 00:00:00
 <sim_end> --> 2019-12-01 00:00:00
 <route_opt> --> 1
 <doesBasinRoute> --> 1
 <dt_qsim> --> 86400
 <hw_drain_point> --> 2
 <restart_write> --> last
 <restart_date> --> yyyy-mm-dd hh:mm:ss
 <fname_state_in> --> coldstart
 <is_lake_sim> --> F
 <is_flux_wm> --> F
 <is_vol_wm> --> F
 <is_vol_wm_jumpstart> --> F
 <ancil_dir> --> ancillary_data/
 <input_dir> --> input/
 <output_dir> --> output/
 <fname_ntopOld> --> rhone_rivertopo.nc
 <dname_sseg> --> seg
 <dname_nhru> --> hru
 <fname_qsim> --> total_runoff_rhone_watergap.nc
 <vname_qsim> --> qtot
 <vname_precip> --> VARNAME_PRECIP
 <vname_evapo> --> VARNAME_EVAPO
 <vname_time> --> time
 <vname_hruid> --> ID
 <dname_time> --> time
 <dname_hruid> --> ID
 <units_qsim> --> mm/s
 <ro_time_units> --> month
 <ro_time_stamp> --> start
 <fname_wm> --> WM_NC_IN
 <vname_flux_wm> --> VARNAME_WM_FLUX
 <vname_vol_wm> --> VARNAME_WM_VOL
 <vname_time_wm> --> VARNAME_WM_TIME
 <vname_segid_wm> --> VARNAME_WM_SEG
 <dname_time_wm> --> DIMNAME_WM_TIME
 <dname_segid_wm> --> DIMNAME_WM_SEG
 <is_remap> --> F
 <fname_remap> --> MAPPING_NC
 <vname_hruid_in_remap> --> VARNAME_MAP_RN_HRU
 <vname_weight> --> VARNAME_MAP_WEIGHT
 <vname_qhruid> --> VARNAME_MAP_RO_HRU
 <vname_num_qhru> --> VARNAME_MAP_NUM_RO_HRU
 <dname_hru_remap> --> DIMNAME_MAP_RN_HRU
 <dname_data_remap> --> DIMNAME_MAP_DATA
 <vname_i_index> --> DIMNAME_I_INDEX
 <vname_j_index> --> DIMNAME_J_INDEX
 <hydGeometryOption> --> 0
 <topoNetworkOption> --> 1
 <computeReachList> --> 1
 <param_nml> --> param.nml
 <varname_area> --> area
 <varname_length> --> length
 <varname_slope> --> slope
 <varname_HRUid> --> HRUid
 <varname_hruSegId> --> hruSegId
 <varname_segId> --> segId
 <varname_downSegId> --> downSegId
 <varname_islake> --> lake
 <varname_lakeModelType> --> lake_type
 <varname_LakeTargVol> --> target_vol
 <seg_outlet> --> -9999
 <ntopAugmentMode> --> F
 <fname_ntopNew> --> augmented_rhone_network.nc
 <debug> --> T
 <desireId> --> -9999
 <newFileFrequency> --> monthly
 <outputFrequency> --> monthly
 <histTimeStamp_offset> --> 0
 <basRunoff> --> T
 <instRunoff> --> T
 <dlayRunoff> --> T
 <sumUpstreamRunoff> --> T
 <KWTroutedRunoff> --> T
 <IRFroutedRunoff> --> T
 <MCroutedRunoff> --> T
 <KWroutedRunoff> --> T
 <DWroutedRunoff> --> T
 <KWTvolume> --> T
 <IRFvolume> --> T
 <KWvolume> --> T
 <MCvolume> --> T
 <DWvolume> --> T

---- calendar --- 
  calendar used for simulation and history output is the same as runoff input
  calendar used in runoff input will be read from total_runoff_rhone_watergap.nc

---- time units --- 
  time_unit for history files will be the same as runoff input
  time_unit in runoff input is provided in control file: month

---- runoff unit --- 
  runoff unit is provided as: mm/s

---- input time stamp --- 
  Input time stamp <ro_time_stamp>:  start
  The same time stamp is used for history output

---- output/simulation time steps --- 
  simulation time step <dt_qsim>:                86400.0
  history file freqeuncy <newFileFrequency>:  monthly
  history output freqeuncy <outputFrequency>: monthly

---- mis. routing options --- 

---- Read river network data --- 
  Reading area into structure HRU
  Reading HRUid into structure HRU2SEG
  Reading hruSegId into structure HRU2SEG
  Reading length into structure SEG
  Reading slope into structure SEG
  Reading segId into structure NTOPO
  Reading downSegId into structure NTOPO

[nmizukami]

Hi, Sorry for the error. This error looks familiar, but I thought this has been fixed, but possibly not. I don't catch anything suspicious for your compiling, setup. This issue is from the computation of river network topology using MERIT data, and not from your runoff input. I have merit network so I will subset to Rhône basin and look into this.

One possibly easy thing for you to try is to use different MPI tasks. How many MPI tasks are you using? you could use just one (so just run it as a serial code) or two. if you are running it on HPC or cluster, you could try increase MPI tasks.

[arthurronner]

Hi! It might have to do with how I try to run the program - I don't use MPI but instead call the executable directly from the slurm file. MPI is installed on the system though. Maybe the number of nodes and tasks do not get properly transfered to MPI? Below is the (very basic) slurm script that I use.

#!/bin/bash
#SBATCH --job-name=test_mizuroute
#SBATCH --output=%x_%j.out
#SBATCH --partition="testing"
#SBATCH --time=00:45:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=2
#SBATCH --mem=1G

echo "### Starting Test"
echo "This is $SLURM_JOB_USER to test mizuRoute, job ID $SLURM_JOB_ID"

echo "This job was submitted from $SLURM_SUBMIT_DIR

module load netCDF-Fortran/4.6.1-gompi-2023a CMake/3.26.3-GCCcore-12.3.0

# Run model with specified control file
/home/ronnerac/mizuRoute/mizuRoute_code/route/bin/route_runoff "/home/ronnerac/mizuRoute/mizuRoute_code/route/settings/SAMPLE.control

[arthurronner]

I increased the number of tasks to 2, which resolved the error! (Thought there is now a timestep error in my input that I need to resolve). Thanks for the quick reply!

[arthurronner]

I celebrated too soon, after fixing the timestep error (inconsistent input forcing due to using a monthly dataset - switched to daily input) and trying to increase the --ntasks argument, I keep running into the same error.

[nmizukami]

Ok, I will try to look today or tomorrow.

[ShervanGharari]

@arthurronner
I share it in case it's helpful.

1- Perhaps an additional check would be to verify that in the ntopo file the segID and nextSegID values are properly constructed. For example, the segID–nextSegID pairs should form a unified and continuous domain, with no breaks, missing links, or hanging ends in the river network topology. This can be checked, for example, with the networkx package of Python.

2- Make sure the last nextSegID for the last SegID is set as -9999.

3- Another (possibly less relevant) suggestion is to confirm that the variable mappings are consistent, i.e.:

<varname_HRUid> → HRUid
<varname_hruSegId> → hruSegId
<varname_segId> → segId

and that these effectively point to the same variables, or three identical variables.

Depending on what Naoki suggests after looking into this, I can also create a new ntopo file for your case, so it can be used for comparison with your ntopo. I need to know the most downstream of your segment, of course.

[nmizukami]

Hi @arthurronner, I tested with my own MERIT network including only Rhône river.

netcdf ntopo_MERIT_Hydro_v1.rhone {
dimensions:
	hru = 2114 ;
	seg = 2114 ;
	pfaf = 32 ;
variables:
	double Basin_Area(hru) ;
		Basin_Area:long_name = "basin area" ;
		Basin_Area:units = "m2" ;
		Basin_Area:_FillValue = -9999. ;
	int hruid(hru) ;
		hruid:long_name = "unique HRU id" ;
		hruid:units = "-" ;
		hruid:_FillValue = -9999 ;
	int hru_seg_id(hru) ;
		hru_seg_id:long_name = "id of the stream segment below each HRU" ;
		hru_seg_id:units = "-" ;
		hru_seg_id:_FillValue = -9999 ;
	double Length(seg) ;
		Length:long_name = "length of segment" ;
		Length:units = "m" ;
		Length:_FillValue = -9999. ;
	double Slope(seg) ;
		Slope:long_name = "slope of segment" ;
		Slope:units = "-" ;
		Slope:_FillValue = -9999. ;
	int seg_id(seg) ;
		seg_id:long_name = "unique id of each stream segment" ;
		seg_id:units = "-" ;
		seg_id:_FillValue = -9999 ;
	int Tosegment(seg) ;
		Tosegment:long_name = "unique id of the next downstream segment" ;
		Tosegment:units = "-" ;
		Tosegment:_FillValue = -9999 ; 

I did not get any error by running with one core (just executed it on terminal like you did). I tried using 2, 3, and 4 cores.

So something may be going on with your network as Shervan said. If you don't mind sharing your network and runoff data, I am happy to look. You can send it to my email (mizukami@ucar.edu).

[nmizukami]

Hi @arthurronner,
thanks for the data. I tested with your data on my Mac, I was able to run it using one task, but with optimized mode (change MODE to fast in Makefile), but if I use MODE=debug, I reproduced your error. I thought I fixed at #552, but I will need to go back and look. I believe you are using gcc compiler?

For you, you can try MODE=fast in Makefile and hope this gets you going.

[arthurronner]

Yes, I am using the gcc compiler

[nmizukami]

Ok. One error I found in rhone_rivertopo.nc.

The outlet reach of rhone river: segId=21005420. This should link to a catchment hru (hruId=21005420). So hruSegId for this hru should be 21005420, but now it is missing values (-9999). So now, outlet hru 21005420 is not draining into any river reach (this situation may be valid for some - there are some hrus that are not connected to any rivers, but not for your case). I believe if you fix this, you should be able to run.

But going through this, I found some logic error in handing hrus not connected to rivers during domain decomposition, so I will have to fix the bug.

[arthurronner]

Thank you! Fixed this issue in the river topology, and now the error is gone. Another error took its place though, will open another discussion to highlight it.

[nmizukami]

Hi @arthurronner, I am not sure whether your rhone river basin case was intended only for testing purpose, but I believe your network netCDF contains incorrect HRU-reach correspondences (it is variable hruSegId). This variable links each HRU to its corresponding segment ID (dimension is hru). Currently, one reach is connected to at least two HRUs, and many reaches are not connected to any HRUs. For the MERIT-basin network, hruSegId should be the same as hruId (i.e., the ID of a HRU should match the ID of the reach linked to that HRU). I discover this while fixing the bug in domain decomposition using your data set.

[arthurronner]

Hi @nmizukami, Thank you, after your and Shervan's comments a couple of weeks ago I discovered this as well. That was the error in my topology in the first place. I fixed it and things are running as intended for the Rhône basin (and larger basins by now).

[arthurronner]

My misunderstanding was because I mixed up the meaning of hruSegId. I read the description as 'the segment that the hru flows into' as the downstream segment of the hru - not the segment that the hru is connected to.

[nmizukami]

Summary:

Although rhone_rivertopo.nc has the error in outlet hru (the hru was not connected to river reach), the code was supposed to run. There is a bug in the code and PR #572 should fix this.