Skip to content

Conversation

@BenjaminTJohnson
Copy link
Contributor

Description

This PR updates the CRTM to use NetCDF as the default format for all Look-Up Table (LUT) interfaces (Aerosol, Cloud, Spectral, Transmittance, and Emissivity), replacing the legacy Binary default. This aligns the codebase with modern data standards while maintaining robust backward compatibility for legacy binary datasets.

Key Changes

  1. Default Format Switch:

    • Updated CRTM_LifeCycle.f90 to set netCDF as the default format for all coefficient types.
    • Updated default filenames in CRTM_LifeCycle.f90 to point to their .nc4 / .nc equivalents.
    • Refactored low-level IO modules (e.g., SpcCoeff_IO.f90, TauCoeff_IO.f90, AerosolCoeff_IO.f90) to prioritize NetCDF when format flags are unspecified.
  2. Robust Backward Compatibility (Smart Detection):

    • Implemented Smart Format Detection within CRTM_Init. The system now automatically checks for the existence of SpcCoeff/TauCoeff files. If the default (NetCDF) file is missing but a legacy Binary file exists, it seamlessly falls back to Binary format. This ensures that existing user applications and regression tests relying on binary data continue to work without modification.
    • Improved Default_File_Path logic to properly utilize the File_Path argument as a fallback for NetCDF loading when NC_File_Path is not provided.
  3. Test Suite Updates:

    • Updated test_spc_io.f90 to explicitly request netCDF=.FALSE. to preserve the validity of binary IO unit tests.
    • Added test_becoeff_io_nc.f90 to increase coverage for NetCDF BeCoeff IO.
    • Streamlined test/CMakeLists.txt configuration.
    • Verified that all 193 CTests pass, covering both new NetCDF workflows and legacy Binary regressions.
  4. New Utilities:

    • Added TauCoeff_Inspect tool for inspecting Transmittance Coefficient files.

Testing

  • ctest -j4: 100% Passed (193/193 tests).
  • Verified successful softlinking of NetCDF assets in the test environment.

@BenjaminTJohnson
Copy link
Contributor Author

This requires A LOT of testing and integration checks in FV3-JEDI / UFO workflows.

@BenjaminTJohnson BenjaminTJohnson added the CRTM CRTM label Jan 22, 2026
Default_MWwaterCoeff_File = TRIM(ADJUSTL(File_Path)) // TRIM(Default_MWwaterCoeff_File)
END IF

! Smart detect SpcCoeff format if not specified
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be adopted for all LUTs, to get ride of format specification.

Copy link
Contributor

@chengdang chengdang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this effort!
All tests passed and it looks good to me.

Meanwhile, I wonder if the merge of this PR into dev will perhaps break UFO, could initiate a collaborative PR merge with https://github.com/JCSDA-internal/ufo/pull/3825, in which the coefficient format argument is added in UFO. But, with the smart format detection in CRTM_Lifecycle, we probably won't need these any more.

In any case, ShihWei could probably help test this NetCDF feature in UFO. weiwilliam

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CRTM CRTM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants