Require same datatype for all arrays in multiscales:datasets #154
Conversation
Fixes #2
Automated Review URLs |
will-moore
left a comment
will-moore
left a comment
There was a problem hiding this comment.
Looks good. A couple of minor comments
| to the current zarr group. The "path"s MUST be ordered from largest (i.e. highest resolution) to smallest. | ||
|
|
||
| Each "datasets" dictionary MUST have the same number of dimensions and MUST NOT have more than 5 dimensions. The number of dimensions and order MUST correspond to number and order of "axes". | ||
| All arrays referenced in "datasets" MUST have the same number of dimensions and MUST have the same datatype. The number of dimensions MUST NOT be more than 5 dimensions. |
There was a problem hiding this comment.
"The number of dimensions MUST NOT be more than 5 dimensions" - I think this rule is due to be removed in v0.5? cc@bogovicj I don't know if that was going to happen in #138 but I don't see that it has yet. So it may need a rebase etc if this is merged first to avoid conflicts
|
|
||
| Each "multiscales" dictionary MUST contain the field "datasets", which is a list of dictionaries describing the arrays storing the individual resolution levels. | ||
| Each dictionary in "datasets" MUST contain the field "path", whose value contains the path to the array for this resolution relative | ||
| Each dictionary in "datasets" MUST contain the field "path", whose value contains the path to the array for this resolution relative. |
There was a problem hiding this comment.
Full stop added accidentally? The sentence continues on the next line
| │ │ # by the "multiscales" metadata, but is often a sequence starting at 0. | ||
| │ │ | ||
| │ │ # All arrays must have the same datatype and the same number of dimensions. | ||
| | | |
There was a problem hiding this comment.
Minor point: Looks like 2 lines have changed because whitespace was added to the "blank" line. | | Just adding 1 line is preferable.
|
I suppose one place where this might not be wanted would be in label datasaets where the bottom level is just a voxel array and higher levels are variable length as labels get collapsed together? Storing that bottom level as variable length could potentially be costly. |
|
Just got to this PR and issues
To avoid bloating the queue for 0.6, I've added it to the 1.0 milestone, but if it is included sooner, that seems good too. |
Fixes #2
I think it is important to include this change rather soon. Right now data with different dtypes across the dimensions is technically valid according to the spec, but will cause issues for several implementations.
(It does not work in MoBIE; I am pretty sure the same is true for all tools based on BigDataViewer, and I suspect any tool implemented in a strongly typed language will have issues with this).