dN/dmag test

[yymao]

See LSSTDESC/DC2-production#21 for detail and validation data from HSC.

• code to reduce mock data
• code that works within DESCQA framework
• validation data
• validation criteria

[duncandc]

I am going to start working on this.

[yymao]

Here's a preliminary result from this test, made by @duncandc: (and the corresponding entry on DESCQA web interface)

Lines are protoDC2 and buzzard_test, points are HSC. Difference in normalization may be due to a bug (still be explored).

See also LSSTDESC/DC2-production#21 for related comments.

[rmandelb]

@yymao @duncandc - There has been further discussion of this test in #50 (see specifically from #50 (comment) onwards). A few outstanding questions:

• as discussed there, we'd like to have some way of disabling this test for catalogs that have redshift cuts such that the N(<mag) calculation is significantly affected. Can this be implemented based on the maximum redshift of the light cone?
• the last status of this issue was that there were bugs to work out. Is this still in progress?
• validation criterion: we had discussed this on constructing validation test for DC2 dN/dmag DC2-production#21 (see more specifically constructing validation test for DC2 dN/dmag DC2-production#21 (comment) ) and the proposal I would like to make is that at limiting magnitudes of 23, 24, and 25 in i-band, we should require 20% agreement in N(<mag). At limiting magnitudes of 26 and 27, we should require 40% agreement. Pinging a few people who may have comments on this: @janewman-pitt-edu , @msimet .

[yymao]

@rmandelb To answer your questions ---

• Yes, that can be easily implemented.
• We have not made much progress since the Sprint Week (but @duncandc can correct me if that's not the case). I am not sure if there was really a bug in getting dN/dmag from the mocks, or just that the way the validation data is constructed is not consistent with how we do it with the mock.
• Those criteria can be implemented. As mentioned above, we need to make sure the validation data is correctly/consistently constructed.

[janewman-pitt-edu]

The criteria Rachel suggests sounds reasonable to me.