diff --git a/latest/index.bs b/latest/index.bs index e2fc6bcf..ed8b6aa4 100644 --- a/latest/index.bs +++ b/latest/index.bs @@ -175,7 +175,309 @@ For this example we assume an image with 5 dimensions and axes called `t,c,z,y,x └── n +Tables {#table-layout} +---------------------- +The following describes the expected layout for tabular data. +OME-NGFF tables are compatible with the [AnnData model](https://github.com/scverse/anndata). +
+. # Root folder, potentially in S3,
+│ # with a flat list of images.
+│
+└── 123.zarr
+ |
+ ├── .zgroup
+ |
+ ├── .zattrs
+ |
+ └── tables # The tables group is a container which holds one or multiple tables that are compatible with AnnData.
+ |
+ │ # The tables group MAY be in the root of the zarr file.
+ ├── .zgroup # The tables group MAY be in root or in another group.
+ |
+ ├── .zattrs # `.zattrs` MUST contain "tables", which lists the keys of the subgroups that are tables. In this case, the only table is "my_table".
+ # hence `.zattrs` should be equal to `{ "tables": [ "my_table" ] }`.
+ |
+ └── my_table
+ │ # The table group MAY be in the root of the zarr file.
+ ├── .zgroup # The table group MAY be in root or in another group.
+ |
+ ├── .zattrs # `.zattrs` MUST contain "type", which is set to `"ngff:region_table"`
+ | # `.zattrs` MUST contain "region", which is the path to the data the table is annotating.
+ | # "region" MUST be a single path (single region) or an array of paths (multiple regions).
+ | # "region" paths MUST be objects with a key "path" and the path value MUST be a string.
+ | # `.zattrs` MUST contain "region_key" if "region" is an array. "region_key" is the key in `obs` denoting which region a given row corresponds to.
+ | # `.zattrs` MAY contain "instance_key", which is the key in `obs` that denotes which instance in "region" the row corresponds to. If "instance_key" is not provided, the values from the `obs` `.zattrs` "_index" key is used.
+ │
+ ├── X # You MAY add an zarr array `X`.
+ │ │ # `X` MUST not be a complex type (i.e., MUST be a single type)
+ │ │ # `X` MAY be chunked as the user desires.
+ │ ├── .zarray
+ │ ├── 0.0
+ │ │ ...
+ │ └── n.m
+ |
+ ├── layers # You MAY add a `layers` group, which contains dense matrices with the same shape as X.
+ │ │
+ │ ├── .zgroup
+ │ ├── .zattrs # `.zattrs` MUST contain `"keys"`, which is an array of the names of the subgroups containing a `layer`.
+ │ │
+ │ └── layer_0 # You MAY add a zarr array for each layer
+ | | # Each layer array MUST have the same shape as X
+ | | # Each layer array SHOULD be chunked the same as X
+ | ├── .zarray
+ | |
+ | ├── 0.0
+ │ │ ...
+ │ └── n.m
+ │
+ ├── obs # You MUST add an obs group container. The obs group holds a table of annotations on the rows in X.
+ │ │ # The rows in obs MUST be index-matched to the rows in X.
+ │ ├── .zgroup
+ │ │
+ │ ├── .zattrs # `.zattrs` MUST contain `"_index"`, which is the name of the column in obs to be used as the index.
+ │ │ # `.zattrs` MUST contain `"column-order"`, which is a list of the order of the non-_index columns.
+ │ │ # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dataframe"` by AnnData.
+ │ │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.2.0"` by AnnData.
+ │ │
+ │ └── col_0 # Each column in the obs table is a 1D zarr array. The rows can be chunked as the user desires.
+ │ ├── .zarray # However, the obs columns SHOULD be chunked in the same way as the rows in X (if present).
+ │ │
+ │ └─ 0
+ ├── var # You MAY add a var group container. The var group holds a table of annotations on the columns in X.
+ | │ # The rows in var MUST be index-matched to the columns in X (if present).
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"_index"`, which is the name of the column in obs to be used as the index.
+ | │ # `.zattrs` MUST contain `"column-order"`, which is a list of the order of the non-_index columns.
+ | │ # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dataframe"` by AnnData.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.2.0"` by AnnData.
+ | │
+ | ├── array_col # Columns in the var table MAY be a 1D zarr array. The rows can be chunked as the user desires.
+ | | ├── .zarray # However, the var columns SHOULD be chunked in the same way as the columns in X.
+ | | │
+ | | └─ 0
+ | |
+ | └── cat_col # Columns in the var table MAY be categorical
+ | ├── .zattrs. # `.zattrs` MUST contain `"encoding-type"`, which is set to `"categorical"` by AnnData.
+ | | # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.2.0"` by AnnData.
+ | |
+ | ├── categories
+ | | ├── .zarray # categories MUST be a 1D zarr array. The rows can be chunked as the user desires.
+ | | |
+ | | └─ 0
+ | ├── codes
+ | | ├── .zarray # codes MUST be a 1D zarr array. The rows can be chunked as the user desires.
+ | | |
+ | | └─ 0
+ | |
+ | ├── null_col # Columns in the var table MAY nullable integer
+ | ├── .zattrs. # `.zattrs` MUST contain `"encoding-type"`, which is set to `"nullable-integer"` by AnnData.
+ | | # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | |
+ | ├── mask
+ | | ├── .zarray # categories MUST be a 1D zarr array. The rows can be chunked as the user desires.
+ | | |
+ | | └─ 0
+ | └── values
+ | ├── .zarray # codes MUST be a 1D zarr array. The rows can be chunked as the user desires.
+ | |
+ | └─ 0
+ |
+ ├── obsm # You MAY add a obsm group comtainer. The obsm group contains arrays that annotate the rows in X.
+ | │ # The rows in each array MUST be index-matched to the rows in X (if present).
+ | |
+ │ ├── .zgroup
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dict"` by AnnData.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | | # `.zattrs` MUST contain `"keys"`, which is an array of the names of the subgroups containing `obsm` arrays.
+ | │
+ │ └── obsm_0 # You MAY add a zarr array for each obsm matrix.
+ | | # Each obsm array MUST have the same number of rows as X.
+ | | # The rows in each obsm array SHOULD be chunked the same as the rows in X.
+ | ├── .zarray
+ | |
+ | ├── 0.0
+ │ │ ...
+ │ └── n.m
+ |
+ ├── varm # You MAY add a varm group comtainer. The varm group contains arrays that annotate the columns in X.
+ | │ # The rows in each array MUST be index-matched to the columns in X (if present).
+ | |
+ │ ├── .zgroup
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dict"` by AnnData.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | | # `.zattrs` MUST contain `"keys"`, which is an array of the names of the subgroups containing `varm` arrays.
+ | │
+ │ └── varm_0 # You MAY add a zarr array for each varm matrix.
+ | | # Each varm array MUST have the same number of rows as columns in X.
+ | | # The rows in each obsm array SHOULD be chunked the same as the columns in X.
+ | ├── .zarray
+ | ├── 0.0
+ │ │ ...
+ │ └── n.m
+ |
+ ├── obsp # You MAY add a obsp group comtainer. The obsp group contains sparse arrays that annotate the rows in X.
+ | │ # The rows in each array MUST be index-matched to the columns in X (if present).
+ | |
+ │ ├── .zgroup
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dict"` by AnnData.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | | # `.zattrs` MUST contain `"keys"`, which is an array of the names of the subgroups containing `obsp` arrays.
+ | │
+ │ └── obsp_0 # You MAY add a zarr group for each obsp array.
+ | | # Each obsp array MUST have the same number of rows as rows in X.
+ | |
+ │ ├── .zgroup
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"csr_matrix"` or `"csc_matrix"` for compressed sparse row and compressed sparse column, respectively.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | | # `.zattrs` MUST contain `"shape"` which is an array giving the shape of the densified array.
+ | |
+ | ├── data # You MUST add a one-dimensional zarr array named "data".
+ | | | # `data` MAY be chunked as the user desires.
+ | | ├── .zarray
+ | | |
+ | | ├── 0
+ │ │ | ...
+ │ | └── n
+ | |
+ | ├── indices # You MUST add a one-dimensional zarr array named "indices".
+ | | | # `indices` MAY be chunked as the user desires.
+ | | ├── .zarray # `indices` MUST be an `int` dtype.
+ | | |
+ | | ├── 0
+ │ │ | ...
+ │ | └── n
+ | |
+ | └── indptr # You MUST add a one-dimensional zarr array named "indptr".
+ | | # `indptr` MAY be chunked as the user desires.
+ | ├── .zarray # `indptr` MUST be an `int` dtype.
+ | |
+ | ├── 0
+ │ | ...
+ │ └── n
+ |
+ ├── varp # You MAY add a varp group comtainer. The varp group contains sparse arrays that annotate the columns in X.
+ | │ # The rows in each array MUST be index-matched to the columns in X (if present).
+ | |
+ │ ├── .zgroup
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dict"` by AnnData.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | | # `.zattrs` MUST contain `"keys"`, which is an array of the names of the subgroups containing `varp` arrays.
+ | │
+ │ └── varp_0 # You MAY add a zarr group for each varp array.
+ | | # Each varp array MUST have the same number of rows as columns in X.
+ | |
+ │ ├── .zgroup
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"csr_matrix"` or `"csc_matrix"` for compressed sparse row and compressed sparse column, respectively.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | | # `.zattrs` MUST contain `"shape"` which is an array giving the shape of the densified array.
+ | |
+ | ├── data # You MUST add a one-dimensional zarr array named "data".
+ | | | # `data` MAY be chunked as the user desires.
+ | | ├── .zarray
+ | | |
+ | | ├── 0
+ │ │ | ...
+ │ | └── n
+ | |
+ | ├── indices # You MUST add a one-dimensional zarr array named "indices".
+ | | | # `indices` MAY be chunked as the user desires.
+ | | ├── .zarray # `indices` MUST be an `int` dtype.
+ | | |
+ | | ├── 0
+ │ │ | ...
+ │ | └── n
+ | |
+ | └── indptr # You MUST add a one-dimensional zarr array named "indptr".
+ | | # `indptr` MAY be chunked as the user desires.
+ | ├── .zarray # `indptr` MUST be an `int` dtype.
+ | |
+ | ├── 0
+ │ | ...
+ │ └── n
+ |
+ └── uns # You MAY add a uns containter to store unstructured data.
+ |
+ ├── .zgroup
+ |
+ ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dict"` by AnnData.
+ │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ │
+ ├── group # You MAY add zarr groups.
+ | | # `uns` groups MAY contain groups, dataframes, dense arrays, and sparse arrays.
+ | |
+ | ├── .zgroup
+ | |
+ | ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"csr_matrix"` by AnnData.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | ...
+ |
+ ├── dataframe_0 # You MAY add dataframe group containers.
+ | | # dataframes MAY be in the `uns` group or in a subgroup.
+ | │
+ | ├── .zgroup
+ | │
+ | ├── .zattrs # `.zattrs` MUST contain `"_index"`, which is the name of the column in obs to be used as the index.
+ | │ # `.zattrs` MUST contain `"column-order"`, which is a list of the order of the non-_index columns.
+ | │ # `.zattrs` MUST contain `"encoding-type"`, which is set to `"dataframe"` by AnnData.
+ | │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.2.0"` by AnnData.
+ | │
+ | └── col_0 # Each column in the obs table is a 1D zarr array.
+ | ├── .zarray # Each columns MUST be chunked the same, but the chunking may be chosen by the user.
+ | │
+ | └─ 0
+ |
+ ├── dense_array # You MAY dense arrays as n n-dimensional zarr arrays.
+ | │ # `dense_array` MUST not be a complex type (i.e., MUST be a single type)
+ | │ # `dense_array` MAY be chunked as the user desires.
+ | | # `dense array` MAY be in the `uns` group or in a subgroup.
+ | |
+ | ├── .zarray
+ | ├── 0.0
+ | │ ...
+ | └── n.m
+ |
+ └── sparse_array # You MAY add sparse arrays as a zarr group for each sparse array.
+ | # sparse arrays MAY be in the `uns` group or in a subgroup.
+ |
+ ├── .zgroup
+ |
+ ├── .zattrs # `.zattrs` MUST contain `"encoding-type"`, which is set to `"csr_matrix"` or `"csc_matrix"` for compressed sparse row and compressed sparse column, respectively.
+ │ # `.zattrs` MUST contain `"encoding-version"`, which is set to `"0.1.0"` by AnnData.
+ | # `.zattrs` MUST contain `"shape"` which is an array giving the shape of the densified array.
+ |
+ ├── data # You MUST add a one-dimensional zarr array named "data".
+ | | # `data` MAY be chunked as the user desires.
+ | ├── .zarray
+ | |
+ | ├── 0
+ │ | ...
+ | └── n
+ |
+ ├── indices # You MUST add a one-dimensional zarr array named "indices".
+ | | # `indices` MAY be chunked as the user desires.
+ | ├── .zarray # `indices` MUST be an `int` dtype.
+ | |
+ | ├── 0
+ │ | ...
+ | └── n
+ |
+ └── indptr # You MUST add a one-dimensional zarr array named "indptr".
+ | # `indptr` MAY be chunked as the user desires.
+ ├── .zarray # `indptr` MUST be an `int` dtype.
+ |
+ ├── 0
+ | ...
+ └── n
+
+
+
High-content screening {#hcs-layout}
------------------------------------