Only keeping relabeled ids

[davidackerman]

I think here you want a copy of fragments rather than a zeros_like array, otherwise I believe all ids that are not updated are set to 0.