Fix(preprocessing): duplicated labels

[hollandjg]

Ensure that floes detected at different scales don't have the same label.

Changes:

• Avoid integer overflow by using cv2.connectedComponentsWithStats, which has a 32-bit integer type for tracking labels
• Ensure that floes from iteration n+1 are labelled starting from the value of the previous highest label highest_label_so_far + 1

Resolutions:

• resolves fix: add min- and max-floe size limits to preprocessing #66
• resolves bug: floes at different scales can obliterate each other, leaving disconnected chunks #68

[cpaniaguam]

A few observations.

[cpaniaguam]

See #66 (comment)

[hollandjg]

We still get some cases, like this one, where there's a disconnected part of the floe (yellow) which has nothing to do with the other floes (greenish) nearby:

... or this one:

(where you can see a disconnected component near the concave bit on the western side of the floe)

[hollandjg]

I've some new examples with the updated code. It turns out that the opening operation itself causes some of these cases to arise.

Example: a floe which has multiple "lobes" straight out of the FSD algorithm prior to cleaning

In this case, we need some kind of cleaning.

The input image with the floe marked (I can't see anything in the true-color image):

How it appears after the FSD algorithm:

With opening:

With opening then cleaning:

With cleaning alone:

Example: a floe which has an extra "lobe" once opening is applied

The input image with the floe marked (I can't see anything in the true-color image):

How it appears after the FSD algorithm:

with opening:

with opening then cleaning:

with cleaning alone:

Proposal: include "new" cleaning, check in paper whether opening is a core part of the algorithm and decide what to do with it

[hollandjg]

From Buckley et. al. https://doi.org/10.5194/tc-18-5031-2024, Figure 2

• It looks to me as though the "opening" step at the end isn't a fundamental part of the algorithm.
• The opening step was in the original: https://github.com/WilhelmusLab/Segmentation_EB/blame/main/Segmentation_EB.ipynb and we don't have the history of that file in GitHub.

I think the question is whether we want that smoothing step to be applied to all the floes at the end. Given Carlos' comment #66 (comment) it might be inadvisable to leave as-is.

If the smoothing is necessary from a scientific point of view, we could:

• Apply opening to each floe individually, but then we may have single pixels where the opening makes it look like it's part of two or more floes. We'd could apply the opening at each floe scale, so that floes detected at smaller scales wouldn't be allowed to chomp bits out of larger floes.
• Get the contour surrounding each floe from the raw (unsmoothed) pixels, and then smooth that contour. There would be some "collisions" between those contours for sure, but the scale of those would hopefully be smaller than the precision we have on the floe edge position.

@danielmwatkins @cpaniaguam Any other ideas? Or preferences?

[hollandjg]

Note that the failing test is unrelated to this PR, and is fixed in

• test(load): remove broken test that files downloaded look identical to those in repo #69

[hollandjg]

I'd like to suggest keeping the "opening" step at the end of the algorithm, and just using the new "cleaning" to remove the extra disconnected parts. That's the smallest change we can make to the algorithm whilst still fixing the problem.
@danielmwatkins @cpaniaguam @ellenbuckley

[danielmwatkins]

This is an interesting case because it is one, like you note, where there really isn't anything there that it should be identifying. I like the idea of adding a minimal cleaning step at the end.

[cpaniaguam]

Thanks for this!

Because the number of different labels in the images might be high, it's probably better to avoid the accumulator pattern in favor of a comprehension. Left a suggestion regarding this.

Looked into this. Decided against the requested changes.