feat: sam2 integration and proposed sam fix #975
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
keejkrej
commented
Jan 19, 2026
keejkrej
commented
- integrates sam2
- creates pytest for sam2 in tests/segm/test_sam2.py
- improved mask overwriting logic (for dealing with overlapping masks), applies for sam as well
- briefly tested in gui
This commit integrates SAM2.1 into Cell-ACDC, mirroring the existing SAM implementation with support for automatic and point-prompted segmentation. New features: - SAM2 model wrapper (cellacdc/models/sam2/acdcSegment.py) - Supports all 4 SAM2.1 variants (Tiny, Small, Base Plus, Large) - Automatic mask generation mode (grid-based prompts) - Point-prompted segmentation mode - 3D z-stack support - Embedding caching for performance - Background removal - GPU acceleration - Model download support (cellacdc/myutils.py) - Added SAM2 model URLs and download functions - Downloads from official Facebook Research repository - Supports all 4 model variants (~870MB total) - Test infrastructure (tests/segm/) - Automated test for SAM2 with MIA_KC_htb1_mCitrine dataset - Contrast normalization for low-dynamic-range images - Visualization output (3-panel overlay plot) - GPU-accelerated testing - Test data class (cellacdc/data.py) - Added MIA_KC_htb1_mCitrine for budding yeast timelapse data Key improvements over naive integration: - Proper image normalization (stretches intensity range like Fiji auto-contrast) - Optimized parameters for small phase contrast images - Successfully segments yeast cells, buds, and phase contrast halos Test results: 4 objects detected on 143x255 phase contrast image in ~7s on GPU Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add check_install_sam2() for user-friendly install prompts (GUI/CLI) - Add _install_sam2() to install from git+https://github.com/facebookresearch/sam2.git - Add segment_anything and sam2 optional dependencies to pyproject.toml - Update tests to sample every 5th frame and auto-download models Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add sam2 to _available_frameworks() CUDA check in myutils.py - Update SAM2 defaults to match sam2 library: - pred_iou_thresh: 0.88 → 0.8 - crop_n_points_downscale_factor: 2 → 1 - min_mask_region_area: 1 → 0 - gpu: False → True - Fix docstring typo "mto" → "to" Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Users can now install cellpose v4+ with `pip install cellacdc[cellpose]` while cellpose3 feature remains for v3.x compatibility. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add [tool.uv] conflicts for mutually exclusive cellpose/cellpose3 extras - Add dev extra with pytest, pytest-cov, matplotlib - Suppress SAM2 _C import warning in pytest config Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add sam2 to embedding prompt checks so users can save and load image embeddings for SAM2, matching the original segment_anything functionality. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Display mask ID at centroid of each segmented instance for easier visual inspection of segmentation results. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sort masks by area descending before label assignment so smaller, more specific masks overwrite larger ones. This prevents a large erroneous mask from destroying valid small cell detections. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously, the largest mask was assigned label 1, then removed by background detection, leaving a gap (no label 1). Now background is identified and filtered out before sorting and labeling, so label 1 correctly refers to the largest non-background object. Also: - Fix label text placement for ring-shaped objects (use closest mask pixel to centroid instead of centroid itself) - Change test frame sampling from every 5th to every 20th frame - Fix SAM1 labels array shape ([:3] -> [:2] for 2D images) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Collaborator
|
Hi @keejkrej, thank you very much for this! I had a quick look, and it looks really nice. I will check it more thoroughly and merge it soon. Thanks again! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.