feat: Add Detection Fusion

[drduhe]

Detection Fusion

Overview

This PR introduces Detection fusion functionality which adds ability to merge overlapping detections in Sam3ProcessorSAM3 codebase.

---

Summary

Implements detection fusion capability that merges overlapping detections based on IoU threshold, reducing duplicate detections and improving output quality.

Changes

1. New Parameter: fuse_detections_iou_threshold

• sam3/model/sam3_image_processor.py: Added optional parameter to Sam3Processor.__init__()
  • Default: None (fusion disabled)
  • When set (e.g., 0.3), enables merging of overlapping detections
  • Fully backward compatible - existing code continues to work unchanged

2. Detection Fusion Implementation

• _fuse_detections() method: New private method implementing fusion logic
  • Algorithm: Uses Union-Find data structure to group overlapping detections
  • IoU Computation: Uses mask_iou() from sam3.perflib.masks_ops to compute pairwise mask IoU
  • Grouping: Detections with IoU > threshold are grouped together
  • Merging Strategy:
    • Masks: Union operation (logical OR) of all masks in group
    • Scores: Maximum score from the group
    • Boxes: Recomputed from merged mask using box_ops.masks_to_boxes()
  • Edge Cases: Handles empty inputs, single detections, and non-overlapping detections gracefully

3. Integration into Processing Pipeline

• Fusion is applied automatically in set_text_prompt() after mask generation
• Only runs when fuse_detections_iou_threshold is set and detections exist
• Preserves all existing functionality when fusion is disabled

4. Comprehensive Test Suite

• tests/test_sam3_image_processor.py: New test file with 260+ lines of tests
  • Tests fusion disabled by default
  • Tests fusion enabled when threshold is set
  • Tests empty input handling
  • Tests single detection (should remain unchanged)
  • Tests non-overlapping detections (should not fuse)
  • Tests overlapping detections (should fuse correctly)
  • Tests multiple overlapping groups
  • Tests score preservation (uses max score)
  • Tests mask merging (union operation)
  • Tests box recomputation from merged masks

5. Test Configuration Updates

• pyproject.toml: Added warning filters for pytest
  • Filters CUDA-related warnings for CPU testing
  • Filters torch.meshgrid indexing warnings
  • Improves test output clarity

Usage Example

# Fusion disabled (default behavior)
processor = Sam3Processor(model, device="cpu", confidence_threshold=0.5)

# Fusion enabled with IoU threshold of 0.3
processor = Sam3Processor(
    model, 
    device="cpu", 
    confidence_threshold=0.5,
    fuse_detections_iou_threshold=0.3  # Merge detections with IoU > 0.3
)

Impact

• ✅ Reduces duplicate/overlapping detections
• ✅ Improves output quality by merging related detections
• ✅ Fully backward compatible (opt-in feature)
• ✅ Well-tested with comprehensive test suite
• ✅ No performance impact when disabled

---

Testing

• ✅ Comprehensive unit tests in tests/test_sam3_image_processor.py
• ✅ Tests cover all edge cases and scenarios
• ✅ Run tests with: pytest tests/test_sam3_image_processor.py

---

Files Changed

• sam3/model/sam3_image_processor.py - Fusion implementation
• tests/__init__.py - Test module initialization
• tests/test_sam3_image_processor.py - Comprehensive test suite
• pyproject.toml - Test configuration updates

---

Backward Compatibility

✅ Fully backward compatible - All changes are additive or opt-in:

• Detection fusion is disabled by default (fuse_detections_iou_threshold=None)
• No API changes to existing functionality

---

Future Considerations

• May want to expose fusion threshold as a runtime parameter in addition to constructor
• Could add visualization utilities to show fusion results