Skip to content

Update to new jzarr 0.4.2#203

Merged
sbesson merged 7 commits intoglencoesoftware:masterfrom
melissalinkert:jzarr-switch
Nov 15, 2023
Merged

Update to new jzarr 0.4.2#203
sbesson merged 7 commits intoglencoesoftware:masterfrom
melissalinkert:jzarr-switch

Conversation

@melissalinkert
Copy link
Copy Markdown
Member

Prompted by ome/ZarrReader#54, see https://github.com/zarr-developers/jzarr.

Discussed briefly with @sbesson. I am not suggesting that we consider this for bioformats2raw 0.7.0, it's more a place to start. Whatever we decide to do here should likely be applied to other repos where we use jzarr (including but not limited to raw2ometiff).

@joshmoore
Copy link
Copy Markdown
Contributor

These are probably the most relevant missing changes:

+  [gs/master~6] Array nesting: store in .zarray if set
+  [gs/master~7] Array nesting: Handle empty zarr arrays
+  [gs/master~8] Array nesting: Add the ability to use N5-style nested layout

@melissalinkert melissalinkert changed the title Update to new jzarr 0.3.7 Update to new jzarr 0.4.0 Jun 28, 2023
Copy link
Copy Markdown
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following a discussion with @melissalinkert, a few comments:

  • the com.bc.zarr:jzarr library should be considered as inactive
  • the cleanup of all intermediate repositories and the ability to use Maven Central is really nice
  • looking nature of the changes between the 0.3.5 and the 0.4.0 changes bcdev/jzarr@0.3.5...zarr-developers:jzarr:0.4.0, I see nothing of particular concerns:
    • essential metadata got added to the POM like the license 👍
    • a few API additions which could be consumed by the converter e.g. the multi-threaded blosc compression
    • the bump of the jackson dependency which is overriden by a more recent versions from ome:formats-gpl anyways
  • at present, https://github.com/zarr-developers/jzarr feels like the best place to discuss & propose new API additions which would be relevant to the needs of this library
  • we will likely need to unify the change across all conversion utilities to avoid a mixture of jzarr

Could we update

// no getter for DimensionSeparator in ZarrArray
// check that the correct separator was used by checking
// that the expected first chunk file exists
assertTrue(output.resolve("0/0/0/0/0/0/0").toFile().exists());
to use the new dimension separator getter available in the new version?

@sbesson
Copy link
Copy Markdown
Member

sbesson commented Sep 23, 2023

👍 maybe let's just remove the outdated test comment and then this should be ready

   // no getter for DimensionSeparator in ZarrArray
    // check that the correct separator was used by checking
    // that the expected first chunk file exists

Next question is whether we want to plan a 0.7.0 release or a release candidate that can be consumed in all the related converter utility PRs adjusting the dependency to use dev.zarr

@melissalinkert
Copy link
Copy Markdown
Member Author

Test comment updated in 609f092. As discussed separately, this PR should not be merged right now, pending investigation of zarr-developers/jzarr#14.

Comment thread build.gradle Outdated
@sbesson sbesson self-requested a review October 31, 2023 21:18
Copy link
Copy Markdown
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retested the last commit and confirmed that:

  • the array metadata of the generated Zarr datasets no longer include the additional numThreads key
  • the Zarr datasets can be opened both using the zarr-python implementation and another Java implementation e.g. raw2ometiff

A few recent maintenance issues (zarr-developers/jzarr#17) are further reinforcing my feeling expressed in #203 (review) i.e. the inactive state of com.bc.zarr:jzarr effectively elevates the dev.zarr:jzarr fork as the new reference Java implementation for the Zarr v2 specification.

@sbesson sbesson changed the title Update to new jzarr 0.4.0 Update to new jzarr 0.4.2 Nov 3, 2023
@sbesson sbesson merged commit b21ed19 into glencoesoftware:master Nov 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants