Open
Conversation
- Updated `write_image_omezarr` and `write_labels_omezarr` functions to accept pixel sizes as float, tuple, or dictionary, allowing for more flexible input formats. - Introduced `_parse_pixel_sizes` helper function to standardize pixel size extraction and validation. - Enhanced metadata extraction in `extract_metadata_tile_nd2` and `extract_metadata_well_nd2` to include pixel size, objective magnification, zoom magnification, and binning information. - Updated `export_omezarr_image` script to read image data from TIFF or raw formats, improving compatibility with different data sources. - Added warnings for potential inconsistencies in pixel size calibration.
- Introduced `conftest.py` to ensure the repository root is included in `sys.path` for test imports. - Updated `test_preprocess.py` to assert required columns in metadata instead of exact counts. - Modified `test_omezarr_exports.py` to check for any Zarr files in the output directory. - Enhanced `write_image_omezarr` to accept new parameters: `coarsening_factor`, `max_levels`, and `is_label`, improving flexibility in image writing. - Added error handling for `max_levels` and `coarsening_factor` to ensure valid values. - Updated metadata handling in `write_image_omezarr` to accommodate label images and ensure proper storage of pixel sizes.
omezarr_writer.py moved under lib/shared
resolves issue with failed labels import into napari
…ss_zarr_v4 Cherry-picked from 1161474 on 79f1eea_preprocess_zarr_v4. - Add dynamic key selection (CONVERT_SBS_KEY/CONVERT_PHENOTYPE_KEY) based on OME_ZARR_ENABLED - IC fields respect IC_EXT (zarr vs tiff) based on config - Downstream rules (sbs.smk, phenotype.smk) use dynamic keys for preprocess inputs
- Add image_to_omezarr.py script that uses convert_to_array + write_image_omezarr - Add convert_sbs_omezarr and convert_phenotype_omezarr rules - Update CONVERT_*_KEY selection to use _omezarr variants when USE_OME_ZARR=True - This allows direct ND2→Zarr conversion, bypassing TIFF intermediates entirely
- Added integration tests for Zarr preprocessing functionality, ensuring nd2_to_zarr conversion produces outputs equivalent to TIFF conversion. - Updated pytest markers to include integration tests. - Modified existing tests to prioritize Zarr format over TIFF where applicable. - Introduced new rules for Zarr conversion in the Snakemake workflow, allowing for flexible output formats based on configuration. - Implemented a script for direct ND2 to standard Zarr conversion, streamlining the preprocessing pipeline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds comprehensive Zarr and OME-Zarr support to Brieflow, enabling cloud-native data formats for improved performance with large datasets and native visualization support in Napari.
Motivation
Large-scale optical pooled screens generate massive amounts of image data. This PR introduces:
This enhancement significantly improves Brieflow's scalability and interoperability with modern microscopy visualization tools while maintaining full backward compatibility with existing TIFF-based workflows.
Related to ongoing efforts to modernize Brieflow's data handling capabilities and support cloud-native workflows.
What is the nature of your change?
Checklist
pyproject.tomlto reflect the change as designated by semantic versioning.ruff checkandruff format.Backward Compatibility
This PR maintains full backward compatibility: