Meteorology for GCHP advection

[lizziel]

Name and Institution

Name: Lizzie Lundgren
Institution: Harvard University

New GCHP feature or discussion

This issue is to discuss current work related to meteorology used in GCHP advection. There are several things that I hope to get into version 14.3.0.

1. Validation of GCHP runs using hourly mass fluxes. All official benchmarks use 3-hourly winds instead. Hourly mass fluxes are available for GEOS-FP at C720 (limited time range) and GEOS-IT at C180 (met option to be available in 14.3.0). Mass fluxes are not available for MERRA2.
2. Implement mass flux regridding update in MAPL. This update from @sdeastham is currently a MAPL PR pending review. The same update needs to be put into our MAPL fork which is an older version of MAPL than what the PR is based on.
3. Document resource constraints when using mass fluxes. See Issue using REGRID_METHOD_CONSERVE_HFLUX reading c180 GEOS-IT data GEOS-ESM/MAPL#2118.
4. The algorithm for computing dry pressure level edges for advection in the GCHPctmEnv gridded component needs an overhaul. We currently (1) sum moisture-corrected total pressure delta across all levels to get surface dry pressure and then (2) construct the 3D dry pressures from the surface dry pressure using Ap and Bp. This method should be compared with a direct computation of 3D dry pressure from 3D total pressure (no reconstruction from surface pressure using Ap/Bp).
5. Add pressure diagnostics in advection. These will appear in HISTORY.rc for gridded component DYNAMICS instead of GCHPchem.
6. Add budget transport diagnostics and/or vertical flux diagnostics per species.

Pinging @sdeastham and @1Dandan who will help with this work.

[1Dandan]

Thanks @lizziel for initiating the discussion. Just to confirm that I can check out the version of 14.2.0-rc.1to use dry pressure instead of total pressure for mass-flux simulations, right?

[lizziel]

Hi @1Dandan. Yes, you can checkout 14.2.0-rc.1 and use the transport tracer simulation. 14.2.0 is still in development because of an issue with full chemistry but the transport tracer simulation should work fine. We are seeing some wonky results for certain tracers in the stratosphere, e.g. SF6, but I think that is okay for the mass flux validation. @sdeastham, correct me if I am wrong on that.

[sdeastham]

That's correct! At this point we're on a fact-finding mission - let's at least start with 14.2.0-rc.1 and see what happens. We know there are issues no matter what we do, so we may as well try and quantify them.

[1Dandan]

Thanks for confirming. Then I'll go head to set up a simulation (intend to run for the year of 2022 at C24) and will let you know when it is finished.

[lizziel]

Sounds good. All of the runs we will do should be the following:

• Transport Tracer simulation
• 1 year duration (Jan 1 2022 - Jan 1 2023)
• Default diagnostics
• Same initial restart file as used for 14.2.0 transport tracer benchmark (GEOSChem.Restart.20190101_0000z.c24.nc4 located here)
• C24 grid resolution
• GEOS-Chem version 14.2.0-rc.1

I will do the wind runs at Harvard and @1Dandan will do the mass flux runs at WashU. Here is a summary of the first runs we will do.

1. @lizziel: "GEOS-FP" option when creating run directory. This uses 3-hourly winds in advection.
2. @1Dandan: "GEOS-FP native data -> Use mass fluxes? yes" option when creating run directory. This uses hourly C720 mass fluxes in advection.

Both of these runs use dry pressure in advection. The run directory differences are all in ExtData.rc and GCHP.rc, with the primary ExtData.rc differences being (1) fields used for advection (see below), and (2) raw files from GMAO versus processed files. The raw files and the processed files contain the same data but the processed files have the vertical level flipped and data concatenated into daily files. Raw files also handle optical depth differently. Here is a breakdown of the primary differences in ExtData.rc.

GEOS-FP processed and winds in advection

 --- Surface pressure, 3-hr instantaneous ---
PS1 hPa N Y 0        none 1.0 PS ./MetDir/%y4/%m2/GEOSFP.%y4%m2%d2.I3.025x03125.nc
PS2 hPa N Y 0;001000 none 1.0 PS ./MetDir/%y4/%m2/GEOSFP.%y4%m2%d2.I3.025x03125.nc

 --- 3D variables, 3-hr instantaneous ---
SPHU1 kg_kg-1 N Y 0        none none QV ./MetDir/%y4/%m2/GEOSFP.%y4%m2%d2.I3.025x03125.nc
SPHU2 kg_kg-1 N Y 0;001000 none none QV ./MetDir/%y4/%m2/GEOSFP.%y4%m2%d2.I3.025x03125.nc

 --- 3D variables, 3-hr averaged ---
UA;VA m_s-1 N Y F0;013000 none none U;V ./MetDir/%y4/%m2/GEOSFP.%y4%m2%d2.A3dyn.025x03125.nc

GEOS-FP native and mass fluxes in advection

MFXC;MFYC Pa_m+2_s-1    N H F0;003000 none  0.6666666 MFXC;MFYC  ./MetDir/../../GEOS_C720/GEOS_FP_Native/Y%y4/M%m2/D%d2/GEOS.fp.asm.tavg_1hr_ctm_c0720_v72.%y4%m2%d2_%h2%n2.V01.nc4 2021-03-11T00:30:00P01:00
CXC;CYC   1             N H F0;003000 none  none CX;CY           ./MetDir/../../GEOS_C720/GEOS_FP_Native/Y%y4/M%m2/D%d2/GEOS.fp.asm.tavg_1hr_ctm_c0720_v72.%y4%m2%d2_%h2%n2.V01.nc4 2021-03-11T00:30:00P01:00
PS1       Pa            N Y  0        none  0.01 PS              ./MetDir/../../GEOS_C720/GEOS_FP_Native/Y%y4/M%m2/D%d2/GEOS.fp.asm.inst_1hr_ctm_c0720_v72.%y4%m2%d2_%h2%n2.V01.nc4 2021-03-11T00:00:00P01:00
PS2       Pa            N Y  0;001000 none  0.01 PS              ./MetDir/../../GEOS_C720/GEOS_FP_Native/Y%y4/M%m2/D%d2/GEOS.fp.asm.inst_1hr_ctm_c0720_v72.%y4%m2%d2_%h2%n2.V01.nc4 2021-03-11T00:00:00P01:00
SPHU1     kg_kg-1       N Y  0        none  none QV              ./MetDir/../../GEOS_C720/GEOS_FP_Native/Y%y4/M%m2/D%d2/GEOS.fp.asm.inst_1hr_ctm_c0720_v72.%y4%m2%d2_%h2%n2.V01.nc4 2021-03-11T00:00:00P01:00
SPHU2     kg_kg-1       N Y  0;001000 none  none QV              ./MetDir/../../GEOS_C720/GEOS_FP_Native/Y%y4/M%m2/D%d2/GEOS.fp.asm.inst_1hr_ctm_c0720_v72.%y4%m2%d2_%h2%n2.V01.nc4 2021-03-11T00:00:00P01:00
UA;VA     m_s-1         N Y F0;013000 none  none U;V             ./MetDir/Y%y4/M%m2/D%d2/GEOS.fp.asm.tavg3_3d_asm_Nv.%y4%m2%d2_%h2%n2.V01.nc4 2014-02-11T01:30:00P03:00

OPTDEP      TAUCLI+TAUCLW 0

Here are the differences in GCHP.rc:

< METEOROLOGY_VERTICAL_INDEX_IS_TOP_DOWN: .false.
< IMPORT_MASS_FLUX_FROM_EXTDATA: .false.
---
> METEOROLOGY_VERTICAL_INDEX_IS_TOP_DOWN: .true.
> IMPORT_MASS_FLUX_FROM_EXTDATA: .true.

I have a few questions for @sdeastham:

1. Are these two runs sufficient to start?
2. Do you see any problems in the config file entries above ?
3. Should we also do a mass flux run with total pressure in advection?
4. Any other comments?

@1Dandan, do you have any questions?

[1Dandan]

Hi @lizziel, yes, I do have some questions.

1. Is there an option I need to turn on to use moisture-corrected mass flux or is it default?
2. I usually spin up for one month, it may not be sufficient if talking about concentrations in stratosphere. For the simulation period, do you want me to start just at Jan-01-2022 and see the evolving of mass conservation?

[lizziel]

Is there an option I need to turn on to use moisture-corrected mass flux or is it default?

Good question. The default is moisture-corrected mass flux, to go along with the default of using dry pressure. Is using moisture-corrected mass flux the best way? I'm not sure. @sdeastham, do you think we should try different permutations of moisture-corrected mass flux and dry/total pressure? If yes, which combinations?

I usually spin up for one month, it may not be sufficient if talking about concentrations in stratosphere. For the simulation period, do you want me to start just at Jan-01-2022 and see the evolving of mass conservation?

The 14.2.0 transport tracer restart file for 2019 was created from a 10-year run using 14.2.0. The GC-Classic restart file at the end was regridded to C24. I think this is sufficient for what we are trying to do (@sdeastham, tell us if you disagree).

[sdeastham]

Assuming that the moisture-corrected mass flux means the mass flux of dry air only (i.e. multiplying the mass flux by 1.0/(1-QV) where QV is taken from the upwind grid cell), then you would want to use the moisture-corrected flux for dry pressure advection and the original (total) flux for total pressure advection. I don't think that any other combination makes sense but happy to discuss!

As for the restart file - I think that's fine. Using a consistent set of initial conditions is all that really matters until we can get to the point where we can do something rigorous in comparison to GEOS. Starting from a high-res file and degrading to the target resolution would be better than starting from a GC-Classic file or low-res file, but I think the differences will be small.

[sdeastham]

As for the questions you posted @lizziel:

1. Are these two runs sufficient to start? Yes, although see 3
2. Do you see any problems in the config file entries above ? Nothing obvious
3. Should we also do a mass flux run with total pressure in advection? I think this is a good idea. We know it's got problems but data will help us in diagnosis
4. Any other comments? Thanks for leading the charge on this!

[lizziel]

Thanks @sdeastham!
There will now be a third run to add to the list. I will summarize all three here to avoid confusion.

1. @lizziel: Dry pressure and 3-hourly winds in advection. "GEOS-FP" option when creating run directory. No manual changes needed.
2. @1Dandan: Dry pressure and moisture-corrected hourly C720 mass fluxes in advection. "GEOS-FP native data -> Use mass fluxes? yes" option when creating run directory. No manual changes needed.
3. @1Dandan: Total pressure and native hourly C720 mass fluxes (not moisure-corrected) in advection. "GEOS-FP native data -> Use mass fluxes? yes" option when creating run directory. Make manual changes in file GCHP.rc:

USE_TOTAL_AIR_PRESSURE_IN_ADVECTION: 1
CORRECT_MASS_FLUX_FOR_HUMIDITY: 0

I am still waiting for the restart file to show up on the Harvard ftp site. I will post here when it is available. I also want to double-check that the total air pressure and native mass flux options look okay before proceeding.

[lizziel]

The 14.2.0 transport tracer restart file to be used for the mass flux runs is now available. Download file GEOSChem.Restart.20190101_0000z.c24.nc4 at http://ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/14.2.0-rc.1/GCHP/TransportTracers/Restarts/.

[lizziel]

I am using the following libraries for the GEOS-FP winds run:

  1) gmp/6.2.1-fasrc01    5) openmpi/4.1.0-fasrc01   9) netcdf-c/4.8.0-fasrc01
  2) mpfr/4.1.0-fasrc01   6) zlib/1.2.11-fasrc01    10) netcdf-fortran/4.5.3-fasrc01
  3) mpc/1.2.1-fasrc01    7) szip/2.1.1-fasrc01     11) flex/2.6.4-fasrc01
  4) gcc/10.2.0-fasrc01   8) hdf5/1.10.7-fasrc01    12) cmake/3.25.2-fasrc01

I will use 96 cores across 2 nodes (48 cores per node) and enable monthly mid-run restart file.