Skip to content

Conversation

@ceblanton
Copy link
Collaborator

@ceblanton ceblanton commented Dec 15, 2025

Describe your changes

  1. Use xarray for combining climatologies instead of cdo merge. cdo merge does not work for more than 100 variables, and even with fewer variables inserts duplicate coordinate variables. It's possible there are slight metadata issues but the resulting climos work fine for analysis scripts
  2. fre app mask-atmos-plevel needs to trigger on pressure_mask = .false. instead of False and update test case. Also, keep the _unmsk legacy behavior trigger, but remove it from the error message so users don't see it.
  3. Add static history files to the regrid task list

Issue ticket number and link #677

Checklist before requesting a review

  • I ran my code
  • I tried to make my code readable
  • I tried to comment my code
  • I wrote a new test, if applicable
  • I wrote new instructions/documentation, if applicable
  • I ran pytest and inspected it's output
  • I ran pylint and attempted to implement some of it's feedback
  • No print statements; all user-facing info uses logging module

@codecov
Copy link

codecov bot commented Dec 16, 2025

Codecov Report

❌ Patch coverage is 76.19048% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.19%. Comparing base (4ec5686) to head (464379a).
⚠️ Report is 46 commits behind head on main.

Files with missing lines Patch % Lines
fre/app/regrid_xy/regrid_xy.py 50.00% 3 Missing ⚠️
fre/app/generate_time_averages/combine.py 83.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #690      +/-   ##
==========================================
- Coverage   85.32%   85.19%   -0.14%     
==========================================
  Files          68       68              
  Lines        4490     4518      +28     
==========================================
+ Hits         3831     3849      +18     
- Misses        659      669      +10     
Flag Coverage Δ
unittests 85.19% <76.19%> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ceblanton ceblanton marked this pull request as ready for review December 17, 2025 14:17
@ilaflott ilaflott self-requested a review December 17, 2025 15:11
@ceblanton
Copy link
Collaborator Author

Not ready yet. There are some more netcdf attributes that need special handling. xarray loses metadata by default

Chris Blanton and others added 4 commits December 22, 2025 16:49
…" which is now users are writing it.

Also, remove the _unmsk note from the error message so folks don't use it
.false. is now replacing False as the variable attribute trigger
Copy link
Member

@ilaflott ilaflott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i had more thoughts

Comment on lines 124 to 135
for static_source_dict in pp_input_static_files:
outfile = output_subdir/f"{date}.{static_source_dict['source']}.nc"
test = xr.load_dataset(outfile)

assert "wet_c" not in test
assert "mister" in test
assert "darcy" in test
assert "wins" in test

assert np.all(test["mister"].values==np.float64(1.0))
assert np.all(test["darcy"].values==np.float64(2.0))
assert np.all(test["wins"].values==np.float64(3.0))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how many static test cases are there? can we be more careful/explicit with checking the static case? We're overloading the variable test here... within a pytest test routine, at a different scope level. Gives me anxiety!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll agree the static test cases here do not give any additional functionality, and kind of confuse things. I would argue that static regrid-xy functionality can remain untested as it's a subset of temporary regrid-xy which is already tested.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to remove this static-flavored regridding testing because it truly adds nothing to the test coverage, and adds some confusion as you point out.

Regridding static files is literally identical to the temporal files all the way to the fregrid call.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... can we not just make a second separate regrid static function?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a unit test, i mean

Chris Blanton added 2 commits January 28, 2026 19:19
- Improve merge-netcdf variable names for clarity
- Promote useful mask-atmos-plevel debug logging message to info
- Rephrase two try/except's as a humble if
@ceblanton ceblanton requested a review from ilaflott January 29, 2026 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants