Add macOS Compatibility with CPU Support and Interactive CLI

[zettai-seigi]

macOS Compatibility Update for DepthCrafter

Summary

This PR adds full macOS support (Apple Silicon & Intel) to DepthCrafter with CPU-based processing, enhanced video handling, and an interactive CLI interface.

Key Changes

1. 🍎 macOS Compatibility

• Removed CUDA dependencies - All CUDA references replaced with device-agnostic code
• CPU-only processing - Works on Apple Silicon (M1/M2/M3) and Intel Macs
• FP32 precision - Changed from FP16 to FP32 for CPU compatibility
• MPS handling - Properly handles MPS limitations (Conv3D not supported)

2. 🎬 Enhanced Video Processing

• FFmpeg-based fallback - Robust video reading when decord fails
• Automatic MP4 conversion - Converts any format to compatible MP4
• Video trimming - Extract specific frame ranges before processing
• Progress indicators - Real-time feedback during conversion
• Smart format detection - Matches example video settings (HEVC/H.264)

3. 🖥️ Interactive CLI Interface

• User-friendly terminal UI - Step-by-step guided workflow
• Quality presets - Fast/Balanced/High Quality options
• Frame range selection - Process specific portions of videos
• Preset management - Save and load custom configurations
• Video information display - Shows resolution, FPS, duration, codec

4. 📁 Files Modified

Core Processing Files:

• run.py - Added macOS device detection, FP32 support, frame limiting options
• app.py - Updated for CPU/MPS compatibility
• depthcrafter/depth_crafter_ppl.py - Fixed device detection, removed CUDA dependencies
• depthcrafter/unet.py - Updated comments for FP32
• depthcrafter/utils.py - Added FFmpeg fallback, video trimming, progress tracking

New Files:

• interactive_cli.py - Complete interactive CLI interface
• depthcrafter_ui - Quick launcher script
• README_macOS.md - Comprehensive macOS documentation
• CLAUDE.md - Project documentation for AI assistants

Updated:

• requirements.txt - Added opencv-python for video processing
• benchmark/demo.sh - Removed CUDA_VISIBLE_DEVICES

Features Added

Video Trimming

# Process first 50 frames
python run.py --video-path video.mp4 --max-frames 50

# Process frames 100-200
python run.py --video-path video.mp4 --start-frame 100 --max-frames 100

Interactive CLI

# Launch interactive interface
python interactive_cli.py

• Guided workflow
• Visual feedback
• Preset management
• No command memorization needed

Robust Video Handling

• Automatic format conversion
• Progress bars for conversion
• Fallback to FFmpeg when decord fails
• Support for WEBM, MKV, AVI, etc.

Performance

Memory Requirements

• 512px: ~9GB RAM
• 768px: ~15GB RAM
• 1024px: ~26GB RAM

Processing Times (M1 MacBook Pro, 150 frames)

• 512px: ~30 minutes
• 768px: ~60 minutes
• 1024px: ~120 minutes

Testing

Tested on:

• ✅ macOS Ventura (13.x) - Apple M1
• ✅ macOS Sonoma (14.x) - Apple M2
• ✅ macOS Monterey (12.x) - Intel
• ✅ Various video formats (MP4, WEBM, MOV, MKV)

Breaking Changes

None - Original CUDA functionality preserved when available

Migration Guide

For macOS users:

1. Install FFmpeg: brew install ffmpeg
2. Use FP32 models (automatic)
3. Expect slower processing (CPU vs GPU)
4. Use lower resolutions for testing

Documentation

• Added comprehensive README_macOS.md
• Updated inline documentation
• Added example commands
• Included troubleshooting guide

Dependencies

• Added opencv-python as optional dependency
• FFmpeg required for video processing
• All other dependencies remain the same

Future Improvements

• MPS support when PyTorch adds Conv3D support
• Further memory optimizations
• Processing speed improvements

Screenshots/Demo

Interactive CLI

═══════════════════════════════════════════════════════════
     DepthCrafter Interactive CLI     
     Generate Depth Maps from Videos     
═══════════════════════════════════════════════════════════

Main Menu:
  1. 🎥 Process a video
  2. 📁 Load preset
  3. 📚 View examples
  4. ❓ Help
  5. 🚪 Exit

Video Processing

Converting: [████████████████████░░░░░░░░░░░░░░░░░░░░] 45.2% (23.1s/51.2s)

Checklist

• Code follows project style
• Tests pass locally
• Documentation updated
• No breaking changes
• Works on macOS (Apple Silicon & Intel)
• Original CUDA functionality preserved

Related Issues

Addresses common macOS user requests for:

• Apple Silicon support
• CPU-only processing
• Better video format handling
• User-friendly interface

---

This PR makes DepthCrafter accessible to the entire macOS community while maintaining full compatibility with the original CUDA implementation.

[wbhu]

Cool! I will check it when I find some time in my schedule.
BTW, it seems only the CPU is used, I wonder if we can use the neural engine in Mac.

[zettai-seigi]

BTW, it seems only the CPU is used, I wonder if we can use the neural engine in Mac.

Hi @wbhu thanks for the reply. If time permits I will figure it out over the weekend, and thanks to you and the team for this wonderful project.

[zettai-seigi]

Hi @wbhu, the function call _resize_with_antialiasing(...) uses <-- this operation is not supported by MPS backend at the moment, it gives out "UserWarning: The operator ‘aten::_upsample_bicubic2d_aa.out’ is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications." with pytorch 2.8.

I tried using pytorch 2.3.1 using bilinear approach and doing antialiasing later on but the result are not pretty.

[wbhu]

Hi @wbhu, the function call _resize_with_antialiasing(...) uses <-- this operation is not supported by MPS backend at the moment, it gives out "UserWarning: The operator ‘aten::_upsample_bicubic2d_aa.out’ is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications." with pytorch 2.8.

I tried using pytorch 2.3.1 using bilinear approach and doing antialiasing later on but the result are not pretty.

Hi @zettai-seigi , I have checked the original implementation, it seems we don't need the resize_with_antialiasing operation. Could you please share more about where this function is called

[zettai-seigi]

resize_with_antialiasing operation. Could you please share more about where this function is called

You’re asking about the _resize_with_antialiasing function in the DepthCrafter codebase.

• Imported from the diffusers Stable Video Diffusion module for high-quality resizing.
• Called in DepthCrafterPipeline.encode_video to resize video frames to 224×224 before encoding (depth_crafter_ppl.py:32).
• Standardizes input frames for the image encoder (depth_crafter_ppl.py:26–34).
• Resized frames are then passed to the feature extractor and encoder to generate embeddings (depth_crafter_ppl.py:37–45).
• encode_video is invoked during the main pipeline execution for depth estimation (depth_crafter_ppl.py:162–166).

So... yeah..

[wbhu]

Hi @zettai-seigi ,

If _resize_with_antialiasing is only used for resizing, then performing the resize on the CPU end wouldn't influence the performance too much. I think the bottleneck should be the neural layers