Working toward running mizuRoute in fully coupled simulations

[ekluzek]

A project needed for water isotopes and our plan to deprecate MOSART and RTM in CESM is to enable running mizuRoute coupled to the MOM ocean model. Beyond enabling it we need to make sure the results for fully coupled simulations are reasonable and don't cause problems in coupled simulations compared to using MOSART.

We have some notes on this in the wiki. But, starting this discussion to work out things that need to be done for this to happen. I'll point to some of the notes in a reply to this main message. Also we'll have a reply that lays out a checklist of things that need to be done.

[ekluzek]

Previous meeting notes:

Meeting with Dave

https://github.com/ESCOMP/mizuRoute/wiki/Meeting-notes:--ctsm-mizuRoute#october-15th-2025-meeting-with-dave

Antarctic network:

https://github.com/ESCOMP/mizuRoute/wiki/Meeting-notes:--ctsm-mizuRoute#aug-21st-2024

Notes from meeting with Mike:

https://github.com/ESCOMP/mizuRoute/wiki/Meeting-notes:--ctsm-mizuRoute#aug-2nd-2023-with-sean-and-mike-levy

[ekluzek]

Task list of things to develop:

• cesm branch/fork for running coupled:

  • Custom sandbox
  • CESM branch -- Erik make a branch that can be shared
  • CTSM branch

• Start with the MOSART grid for mizuRoute for I compsets compare to MOSART

  • Replicate what MOSART does in mizuRoute: f09_t232.I2000Clm60SpMizGs.derecho_intel vs f09_t232.I2000Clm60Sp.derecho_intel
  • A difference is mizuRoute doesn't have HRU's over ocean. We think this might be OK, because MOSART probably only sends to ocean over land.

• Test Existing Coupled compsets:
  We thought there might be issues with the PE layout, but it looks like there won't be.

  • Fully coupled (starting in 1850): (B-case)
    • MOSART (use alias): BHISTC_LTso
    • mizuRoute (use long name): HISTC_CAM70%LT_CLM60%BGC-CROP_CICE_MOM6_MIZUROUTE_DGLC%NOEVOLVE_WW3
    • grid: ne30pg3_t232
    • run both for 5 days
  • Fully coupled (starting in 1850) - same as the above except for 5months:
  • Coupled with CAM/DOCN: (F-case - subset of B-case)
    • MOSART: FHISTC_LTso
    • mizuRoute: 2000_CAM70_CLM50%SP_CICE%PRES_DOCN%DOM_MIZUROUTE_SGLC_SWAV
    • grid: ne30pg3_ne30pg3_mt232, ne16pg3_ne16pg3_mt232, ne3pg2_ne3pg3_mt232
  • Slab Ocean Model: (E-case)
    • MOSART: ETEST
    • mizuRoute:2000_CAM70_CLM50%SP_CICE_DOCN%SOM_MIZUROUTE_SGLC_SWAV
    • grid: f19_f19_mt232, ne30pg3_ne30pg3_mt232, ne30pg3_t232

• Try some custom setups:

  • CAM with MOM setup (CICE prescribed, stub wave): 2000_CAM70_CLM60%SP_CICE%PRES_MOM6_MIZUROUTE_DGLC%NOEVOLVE_SWAV

• Develop some custom setups: (we don't have these now, but good to have)

  • DATM for CLM and MOM same forcing
  • DATM for CLM and MOM different forcing from different tasks for each

[nmizukami]

Erik created a CESM branch to test mizuRoute in fully coupled model - github.com:ekluzek/test_mizuroute

git clone --origin ekluzek -b test_mizuroute git@github.com:ekluzek/CESM.git cesm_mizuRoute

Next step: try I compset to make sure CTSM-mizuRoute/mosart coupling works

[ekluzek]

The initial version of the branch was pointing to a hash that didn't exist. But, it's been updated and is now working.

[ekluzek]

Some issues that apply here:

#526
#469
#183
#153

[nmizukami]

Status as of Nov 7 2025:

Fully coupled
MOSART (use alias): BHISTC_LTso
mizuRoute (use long name): HISTC_CAM70%LT_CLM60%BGC-CROP_CICE_MOM6_MIZUROUTE_DGLC%NOEVOLVE_WW3
grid: ne30pg3_t232

case created and bulit for both MOSART and mizuRoute, but not run yet.

[nmizukami]

Failed HISTC_CAM70%LT_CLM60%BGC-CROP_CICE_MOM6_MIZUROUTE_DGLC%NOEVOLVE_WW3
grid: ne30pg3_t232 (mizuRoute cesm test)

Comparison of med.log file between BHISTC_LTso (mosart cesm test) and mizuRoute cesm test.

mosart cesm test run directory: glade/derecho/scratch/mizukami/SMS_D.ne30pg3_t232.BHISTC_LTso.derecho_intel.20251106_141101_firstd/run

grep -i -w rof med.log.3578963.desched1.251110-095930

 (internalstate init) rof model= mosart
  |from to -med  atm  lnd  ocn  ice  rof  wav  glc1 
  |rof         -    -    T    T    T    -    -    -
  |from to -med  atm  lnd  ocn  ice  rof  wav  glc1 
  |rof         -    -    T    T    T    -    -    -
(med.F90:DataInitialize) initializing FBs for rof
 mapping lnd->rof Flrl_rofsur via consf with lfrac normalization
 mapping lnd->rof Flrl_rofi via consf with lfrac normalization
 mapping lnd->rof Flrl_rofgwl via consf with lfrac normalization
 mapping lnd->rof Flrl_rofsub via consf with lfrac normalization
 mapping lnd->rof Flrl_irrig via consf with lfrac normalization
 mapping rof->lnd Flrr_volr via consf with one normalization
 mapping rof->lnd Flrr_volrmch via consf with one normalization
 mapping rof->lnd Flrr_flood via consf with one normalization
 mapping rof->ocn Flrr_flood via consd with one normalization
 mapping rof->ocn Forr_rofl via rof2ocn_liq with none normalization
 mapping rof->ocn Forr_rofi via rof2ocn_ice with none normalization
 mapping rof->ocn Forr_rofl_glc via rof2ocn_liq with none normalization
 mapping rof->ocn Forr_rofi_glc via rof2ocn_ice with none normalization
 mapping glc1->rof Fgrg_rofi via consd with gfrac normalization
(merge_to_lnd): Flrr_volr(lnd) = Flrr_volr(rof)
(merge_to_lnd): Flrr_volrmch(lnd) = Flrr_volrmch(rof)
(merge_to_lnd): Flrr_flood(lnd) = Flrr_flood(rof)
(merge_to_ocn): Foxx_rofl(ocn) = Forr_rofl:Flrr_flood(rof)
(merge_to_ocn): Foxx_rofi(ocn) = Forr_rofi(rof)
(merge_to_ocn): Forr_rofl_glc(ocn) = Forr_rofl_glc(rof)
(merge_to_ocn): Forr_rofi_glc(ocn) = Forr_rofi_glc(rof)
(merge_to_rof): Flrl_rofsur(rof) = lfrac*Flrl_rofsur(lnd)
(merge_to_rof): Flrl_rofi(rof) = lfrac*Flrl_rofi(lnd)
(merge_to_rof): Flrl_rofgwl(rof) = lfrac*Flrl_rofgwl(lnd)
(merge_to_rof): Flrl_rofsub(rof) = lfrac*Flrl_rofsub(lnd)
(merge_to_rof): Flrl_irrig(rof) = lfrac*Flrl_irrig(lnd)
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consf for lnd to rof srcMask =    -987987 dstMask =    -987987
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consf for rof to lnd srcMask =    -987987 dstMask =    -987987
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consd for rof to ocn srcMask =    -987987 dstMask =          0
 (module_med_map: med_map_routehandles_initfrom_field) creating RH rof2ocn_liq via input file /glade/campaign/cesm/cesmdata/inputdata/cpl/gridmaps/r05/map_r05_to_tx2_3_nnsm_e250r250_230914.nc for rof to ocn srcMask =    -987987 dstMask =          0
 (module_med_map: med_map_routehandles_initfrom_field) creating RH rof2ocn_ice via input file /glade/campaign/cesm/cesmdata/inputdata/cpl/gridmaps/r05/map_r05_to_tx2_3_nnsm_e1000r1000_230415.nc for rof to ocn srcMask =    -987987 dstMask =          0
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consd for glc1 to rof srcMask =    -987987 dstMask =    -987987
 (med_map_mod: RouteHandles_init) created field_NormOne for lnd->rof with mapping consf
 (med_map_mod: RouteHandles_init) created field_NormOne for rof->lnd with mapping consf
 (med_map_mod: RouteHandles_init) created field_NormOne for rof->ocn with mapping consd
 (med_map_mod: RouteHandles_init) created field_NormOne for rof->ocn with mapping rof2ocn_ice
 (med_map_mod: RouteHandles_init) created field_NormOne for rof->ocn with mapping rof2ocn_liq
 (med_map_mod: RouteHandles_init) created field_NormOne for glc1->rof with mapping consd
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_irrig       1
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofgwl       2
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofi       3
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsub       4
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsur       5
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consd  gfrac  Fgrg_rofi       1
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_irrig       1
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofgwl       2
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofi       3
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsub       4
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsur       5
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consd for lnd to rof srcMask =    -987987 dstMask =    -987987
(med.F90:DataInitialize) rof T T
global nx,ny,ntile sizes for rof:     720     360       0

mizuRoute run directory: /glade/derecho/scratch/mizukami/SMS_D.ne30pg3_t232.HISTC_CAM70%LT_CLM60%BGC-CROP_CICE_MOM6_MIZUROUTE_DGLC%NOEVOLVE_WW3.derecho_intel.20251106_141131_yedazb/run

grep -i -r -w rof med.log.3579934.desched1.251110-124915

 (internalstate init) rof model= mizuroute
  |from to -med  atm  lnd  ocn  ice  rof  wav  glc1 
  |rof         -    -    T    T    T    -    -    -
  |from to -med  atm  lnd  ocn  ice  rof  wav  glc1 
  |rof         -    -    T    T    T    -    -    -
(med.F90:DataInitialize) initializing FBs for rof
 mapping lnd->rof Flrl_rofsur via consf with lfrac normalization
 mapping lnd->rof Flrl_rofi via consf with lfrac normalization
 mapping lnd->rof Flrl_rofgwl via consf with lfrac normalization
 mapping lnd->rof Flrl_rofsub via consf with lfrac normalization
 mapping lnd->rof Flrl_irrig via consf with lfrac normalization
 mapping rof->lnd Flrr_volr via consf with one normalization
 mapping rof->lnd Flrr_volrmch via consf with one normalization
 mapping rof->lnd Flrr_flood via consf with one normalization
 mapping rof->ocn Flrr_flood via consd with one normalization
 mapping rof->ocn Forr_rofl via consd with one normalization
 mapping rof->ocn Forr_rofi via consd with one normalization
(merge_to_lnd): Flrr_volr(lnd) = Flrr_volr(rof)
(merge_to_lnd): Flrr_volrmch(lnd) = Flrr_volrmch(rof)
(merge_to_lnd): Flrr_flood(lnd) = Flrr_flood(rof)
(merge_to_ocn): Foxx_rofl(ocn) = Forr_rofl:Flrr_flood(rof)
(merge_to_ocn): Foxx_rofi(ocn) = Forr_rofi(rof)
(merge_to_rof): Flrl_rofsur(rof) = lfrac*Flrl_rofsur(lnd)
(merge_to_rof): Flrl_rofi(rof) = lfrac*Flrl_rofi(lnd)
(merge_to_rof): Flrl_rofgwl(rof) = lfrac*Flrl_rofgwl(lnd)
(merge_to_rof): Flrl_rofsub(rof) = lfrac*Flrl_rofsub(lnd)
(merge_to_rof): Flrl_irrig(rof) = lfrac*Flrl_irrig(lnd)
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consf for lnd to rof srcMask =    -987987 dstMask =    -987987
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consf for rof to lnd srcMask =    -987987 dstMask =    -987987
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consd for rof to ocn srcMask =    -987987 dstMask =          0
 (med_map_mod: RouteHandles_init) created field_NormOne for lnd->rof with mapping consf
 (med_map_mod: RouteHandles_init) created field_NormOne for rof->lnd with mapping consf
 (med_map_mod: RouteHandles_init) created field_NormOne for rof->ocn with mapping consd
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_irrig       1
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofgwl       2
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofi       3
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsub       4
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsur       5
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_irrig       1
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofgwl       2
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofi       3
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsub       4
 (med_map_mod:med_packed_field_create)Packed field: destcomp,mapping,mapnorm,fldname,index:   rof  consf  lfrac  Flrl_rofsur       5
 (module_med_map: med_map_routehandles_initfrom_field) creating RH consd for lnd to rof srcMask =    -987987 dstMask =    -987987
(med.F90:DataInitialize) rof T T
global nx,ny,ntile sizes for rof:   99427       1       0

[nmizukami]

The above errors in B-case (mizuRoute) have been resolved now