Skip to content

Warn about geographic CRS in raster_band_percentile#429

Merged
dcdenu4 merged 4 commits intonatcap:mainfrom
megannissel:feature/299-raster-band-percentile
Apr 28, 2025
Merged

Warn about geographic CRS in raster_band_percentile#429
dcdenu4 merged 4 commits intonatcap:mainfrom
megannissel:feature/299-raster-band-percentile

Conversation

@megannissel
Copy link
Copy Markdown
Contributor

Fixes #299

  • Update docstring to specify percentiles are calculated based on pixel values
  • Check if raster CRS is geographic; if so, log a warning
  • Add a test case to check logging

@megannissel megannissel marked this pull request as ready for review April 11, 2025 19:04
@dcdenu4 dcdenu4 self-requested a review April 11, 2025 19:23
Copy link
Copy Markdown
Member

@dcdenu4 dcdenu4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @megannissel , just a quick variable name suggestion.

@megannissel megannissel requested a review from dcdenu4 April 11, 2025 20:32
Copy link
Copy Markdown
Member

@phargogh phargogh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just wanted to add a suggested change, and I also had one other thought that I wanted to pose (and @dcdenu4, I'd love your input on this as well): aren't there cases where a raster with a geographic coordinate system is perfectly legitimate? I think this warning is a nice feature for some use cases, but I really don't think we can always assume that pixels have area-dependent values. Example: InVEST Scenic Quality has a value raster (admittedly in a projected coordinate system) that is segmented by percentiles, without consideration of value per unit area.

If you both agree, perhaps we could do the following?

  1. update the language of the warning to eliminate the presumption that all pixel values are per unit area
  2. allow the warning to be disabled with an optional parameter

Sorry to jump in here!

Comment on lines +711 to +719
base_raster = gdal.OpenEx(base_raster_path_band[0], gdal.OF_RASTER)
srs = base_raster.GetSpatialRef()
if srs.IsGeographic():
LOGGER.warning(
f'Raster {base_raster_path_band[0]} has a geographic CRS (pixels '
'do not have equal area). Because `raster_band_percentile` calculates '
'percentiles of pixel values, percentile results will be skewed.')
base_raster = None

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to interject here, but I just found out about gdal.Dataset having the attributes of a context manager! Doing something like the below would help to avoid a case where an error might be raised while checking the spatial reference such as if no spatial reference is defined and None is returned, and the raster should then be closed. If this were to happen, then, at least on Windows a test case would not be able to remove the file because it would still be opened. This way, the raster should be closed when the context manager exits, no matter what happens.

Suggested change
base_raster = gdal.OpenEx(base_raster_path_band[0], gdal.OF_RASTER)
srs = base_raster.GetSpatialRef()
if srs.IsGeographic():
LOGGER.warning(
f'Raster {base_raster_path_band[0]} has a geographic CRS (pixels '
'do not have equal area). Because `raster_band_percentile` calculates '
'percentiles of pixel values, percentile results will be skewed.')
base_raster = None
with gdal.OpenEx(base_raster_path_band[0], gdal.OF_RASTER) as base_raster:
srs = base_raster.GetSpatialRef()
if srs.IsGeographic():
LOGGER.warning(
f'Raster {base_raster_path_band[0]} has a geographic CRS (pixels '
'do not have equal area). Because `raster_band_percentile` calculates '
'percentiles of pixel values, percentile results will be skewed.')

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that's neat; I didn't know you could use it as a context manager!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @phargogh ! I was looking at this as well. Am I right in thinking that this was introduced in GDAL 3.8 and that if we wanted to go this route we'd want to update our requirements for GDAL version?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oooo good catch @dcdenu4 ! Yeah, gdal 3.8 seems like a pretty recent version for this, as much as I want to be abele to use it. I'm not sure we should bump the requirement that high just yet, so @megannissel here's an alternative that doesn't use a context manager:

Suggested change
base_raster = gdal.OpenEx(base_raster_path_band[0], gdal.OF_RASTER)
srs = base_raster.GetSpatialRef()
if srs.IsGeographic():
LOGGER.warning(
f'Raster {base_raster_path_band[0]} has a geographic CRS (pixels '
'do not have equal area). Because `raster_band_percentile` calculates '
'percentiles of pixel values, percentile results will be skewed.')
base_raster = None
try:
base_raster = gdal.OpenEx(base_raster_path_band[0], gdal.OF_RASTER)
srs = base_raster.GetSpatialRef()
if srs.IsGeographic():
LOGGER.warning(
f'Raster {base_raster_path_band[0]} has a geographic CRS (pixels '
'do not have equal area). Because `raster_band_percentile` calculates '
'percentiles of pixel values, percentile results will be skewed.')
finally:
base_raster = None

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @phargogh! I think the question still remains about whether we think this warning is a good idea at all, or if we want to simply stick with the updated docstring. I don't have a strong opinion either way; what do you think?

@megannissel
Copy link
Copy Markdown
Contributor Author

megannissel commented Apr 14, 2025

Thank you for jumping in and explaining a potential caveat, @phargogh! I had taken it on face value that this would always be a concern with a geographic CRS. Assuming @dcdenu4 also agrees, I'm on board with adding an optional parameter to disable the warning. As for re-wording the warning itself, how does this sound?
Raster {filepath} has a geographic CRS. Because `raster_band_percentile` calculates percentiles of pixel values, percentile results may be skewed.

Copy link
Copy Markdown
Member

@dcdenu4 dcdenu4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @megannissel , I just had a question of whether we need to update our gdal dependency requirement for @phargogh suggestion of using a context manager, which I completely agree would be great!

@dcdenu4
Copy link
Copy Markdown
Member

dcdenu4 commented Apr 14, 2025

aren't there cases where a raster with a geographic coordinate system is perfectly legitimate? I think this warning is a nice feature for some use cases, but I really don't think we can always assume that pixels have area-dependent values.

Yes, I don't think it's always the case that this warning is warranted and that there are use cases where the values of the raster are pixel area agnostic.

update the language of the warning to eliminate the presumption that all pixel values are per unit area

Agree! It's more about what THIS function does in its operation on the input. This function operates on pixel values without consider pixel area.

allow the warning to be disabled with an optional parameter

The more I think about this the more I'm torn whether we need to be warning at all and that the docstring update suffices for clarification to PGP users. I don't think there is harm in adding the optional warning, but maybe unnecessary... Thoughts @megannissel and @phargogh ?

@davemfish davemfish added this to the 2.4.8 milestone Apr 22, 2025
@phargogh
Copy link
Copy Markdown
Member

The missing piece of information, from my perspective, is a clear understanding of what the units of the raster are. Until this metadata is available to pygeoprocessing (proposal, anyone? 😉 ) I do not think we can safely make assumptions about the nature of the units and whether the warning is relevant.

However, as programmers, I could see this warning being useful. If I'm writing a scientific workflow, I know things about my inputs and outputs that pygeoprocessing does not, and I absolutely would use a warning like this to help guard against me using the function in the wrong way. So I'd suggest leaving the warning in place, but disabling the warning by default. Just my $0.02.

@megannissel megannissel requested review from dcdenu4 and phargogh April 23, 2025 22:12
Copy link
Copy Markdown
Member

@phargogh phargogh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @megannissel !

Copy link
Copy Markdown
Member

@dcdenu4 dcdenu4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @megannissel. I like @phargogh rationale in his last comment and like this solution. Looks like there are some conflicts, but happy to merge once those are fixed.

@megannissel megannissel requested a review from dcdenu4 April 28, 2025 14:10
@dcdenu4 dcdenu4 merged commit 717323c into natcap:main Apr 28, 2025
63 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clarify raster_band_percentile docstring and warn if raster isn't projected

4 participants