Feature/copernicus pr1 core

[rfievet]

Summary

This PR adds the foundational Copernicus Data Space Ecosystem client for fetching Sentinel-2 satellite imagery. This is the first of 3 PRs to add complete Copernicus support to Galileo as an alternative data source to Google Earth Engine.

What's Included

Core Infrastructure

• OAuth2 Authentication: Client credentials flow with automatic token refresh and 5-minute expiry buffer
• Caching System: Deterministic cache keys for search results and downloaded files with automatic reuse
• Input Validation: Comprehensive validation for bounding boxes, dates, resolution, and cloud cover parameters
• Robust Downloads: HTTP streaming with retry logic, exponential backoff, and resume capability using Range requests
• Session Management: Connection pooling via requests.Session for improved performance

S2 Functionality

• Product Search: OData API queries with spatial/temporal/cloud cover filtering
• Product Download: Complete S2 product downloads (500MB-1GB per product) with progress tracking
• Metadata Creation: JSON metadata files for products when download is disabled
• Interactive Mode: User confirmation prompts before large downloads

Supporting Files

• .env.example - Template for Copernicus credentials
• tests/test_copernicus_s2_basic.py - 18 unit tests with mocked API calls
• tests/manual_test_s2_download.py - Manual integration test script

Architecture

src/data/copernicus/
├── client.py          # Main client with OAuth and S2 support
├── s2.py              # S2-specific search and download logic
├── utils.py           # Validation and helper functions
├── download_utils.py  # Robust download with retry/resume
└── __init__.py        # Module exports

The client uses a modular design where:

• CopernicusClient handles authentication and coordination
• s2.py contains all S2-specific logic
• download_utils.py provides reusable download functionality
• utils.py contains shared validation and conversion functions

Usage Example

from src.data.copernicus import CopernicusClient

# Initialize client (reads credentials from .env)
client = CopernicusClient()

# Fetch Sentinel-2 data
s2_files = client.fetch_s2(
    bbox=[6.15, 49.11, 6.16, 49.12],
    start_date="2024-01-01",
    end_date="2024-01-31",
    max_cloud_cover=20.0,
    product_type="S2MSI2A",  # Level-2A (atmospherically corrected)
    download_data=True,
    max_products=3
)

print(f"Downloaded {len(s2_files)} S2 products")

Testing

Unit Tests (18/18 passing)

• Input validation (bbox, dates, resolution, cloud cover)
• Cache key generation and determinism
• OAuth token flow (mocked)
• Error handling and edge cases
• S2 search query construction

Integration Tests (Verified)

• ✅ Real OAuth2 authentication with Copernicus API
• ✅ Product search returning actual S2 products
• ✅ Complete download of 733MB S2 product
• ✅ ZIP file integrity verified (82 files including spectral bands)
• ✅ Caching working correctly (instant subsequent requests)
• ✅ Download speed: 20.6 MB/s with connection pooling

Test Coverage

# Run unit tests
uv run pytest tests/test_copernicus_s2_basic.py -v

# Run manual integration test (requires credentials)
python tests/manual_test_s2_download.py           # Search only
python tests/manual_test_s2_download.py --download # With download

Dependencies Added

• python-dotenv>=1.0.0 - For loading .env configuration files
• requests>=2.31.0 - For HTTP requests (was only type stubs before)

What's NOT in This PR

As specified in the PR split plan, this PR intentionally excludes:

• ❌ Sentinel-1 support (coming in PR 817 uv dependency support #3)
• ❌ Image processing (RGB extraction, cropping - coming in PR improve CI/CD (coverage, tests, python version etc) #2)
• ❌ Visualization functions (coming in PR improve CI/CD (coverage, tests, python version etc) #2)
• ❌ Spectral indices (NDVI, NDWI, etc. - coming in PR improve CI/CD (coverage, tests, python version etc) #2)
• ❌ Cloud masking and quality assessment (coming in PR improve CI/CD (coverage, tests, python version etc) #2)

Breaking Changes

None. This is a new feature that doesn't affect existing functionality.

Credentials Setup

Users need free Copernicus credentials:

1. Register at https://dataspace.copernicus.eu/
2. Create OAuth client credentials
3. Add to .env file:

COPERNICUS_CLIENT_ID=your_client_id
COPERNICUS_CLIENT_SECRET=your_client_secret

Next Steps

• PR feat: add S2 image processing and visualization #5 : Add S2 image processing (RGB extraction, spectral indices, cloud masking, visualization)
• PR feat: add Sentinel-1 SAR support and integration tests #7 : Add Sentinel-1 SAR support with all acquisition modes

Related Issues

Part of the Copernicus Data Space Ecosystem integration to provide an alternative to Google Earth Engine for data fetching.