Skip to content

Conversation

@hadv
Copy link
Owner

@hadv hadv commented Sep 4, 2025

🚀 Azure Blob Storage Integration for NebulaStore AFS

This PR adds comprehensive Azure Blob Storage support to NebulaStore's Abstract File System (AFS), enabling seamless cloud storage integration alongside the existing AWS S3 connector.

Key Features

  • Full AFS Integration: Complete implementation of IBlobStoreConnector interface
  • Flexible Authentication: Support for both connection strings and Azure managed identity
  • Configuration Builder: Fluent API for easy setup and configuration
  • Path Validation: Azure-specific container and blob name validation
  • Error Handling: Comprehensive exception handling with Azure-specific error mapping
  • NuGet Package: Properly configured as a distributable package

🏗️ Architecture

Core Components

  • AzureStorageConnector: Main connector implementing full CRUD operations
  • AzureStorageConfiguration: Configuration management with builder pattern
  • AzureStoragePathValidator: Azure-specific path validation
  • AzureStorageClientFactory: Centralized Azure client management
  • AzureStorageException: Custom exception handling

Integration Points

  • Seamless compatibility with BlobStoreFileSystem
  • Works with existing AfsStorageConnection
  • Follows established NebulaStore AFS patterns

📦 Dependencies

  • Azure.Storage.Blobs (v12.19.1)
  • Azure.Identity (v1.12.0)
  • NebulaStore.Afs.Blobstore (project reference)

🔧 Usage Example

// Basic usage with connection string
using var connector = AzureStorageConnector.FromConnectionString(
    "DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey");
using var fileSystem = BlobStoreFileSystem.New(connector);

// Advanced configuration
var config = AzureStorageConfiguration.New()
    .SetConnectionString(connectionString)
    .SetUseCache(true)
    .SetTimeout(60000)
    .SetRetryPolicy(maxRetries: 5, retryDelayMs: 2000)
    .Build();

using var connector = AzureStorageConnector.New(config);
using var fileSystem = BlobStoreFileSystem.New(connector);

// Standard file operations
var path = BlobStorePath.Of("my-container", "my-file.txt");
fileSystem.CreateFile(path);
fileSystem.WriteData(path, new[] { Encoding.UTF8.GetBytes("Hello Azure!") });
var data = fileSystem.ReadData(path, 0, -1);

🧪 Testing

  • ✅ Solution builds successfully
  • ✅ All existing tests pass
  • ✅ Azure Storage project compiles without errors
  • ✅ Integration with existing AFS architecture verified

📁 Files Added

afs/azure/storage/
├── NebulaStore.Afs.Azure.Storage.csproj
├── README.md
└── src/
    ├── AzureStorageConnector.cs
    ├── AzureStorageConfiguration.cs
    ├── AzureStoragePathValidator.cs
    ├── AzureStorageClientFactory.cs
    └── AzureStorageAfsIntegration.cs
examples/
└── AzureStorageExample.cs

🎯 Benefits

  1. Multi-Cloud Support: Expands NebulaStore beyond AWS to Azure ecosystem
  2. Enterprise Ready: Supports Azure managed identity for secure authentication
  3. Scalable: Leverages Azure Blob Storage's global scale and performance
  4. Consistent API: Same interface as existing storage connectors
  5. Production Ready: Comprehensive error handling and validation

🔄 Backward Compatibility

  • ✅ No breaking changes to existing APIs
  • ✅ Existing AWS S3 and local storage connectors unaffected
  • ✅ All existing functionality preserved

📋 Checklist

  • Implementation follows NebulaStore coding standards
  • Comprehensive XML documentation added
  • Project properly configured as NuGet package
  • Integration with existing AFS architecture
  • Usage examples provided
  • README documentation updated
  • Solution builds successfully
  • No breaking changes introduced

🚀 Ready for Review

This implementation provides a solid foundation for Azure Blob Storage integration in NebulaStore. The code follows established patterns, includes comprehensive error handling, and maintains full compatibility with the existing ecosystem.

Ready for peer review and testing with Azure storage accounts!


Pull Request opened by Augment Code with guidance from the PR author

- Implement AzureStorageConnector with full IBlobStoreConnector interface
- Add AzureStorageConfiguration with builder pattern for flexible setup
- Include AzureStoragePathValidator for Azure-specific path validation
- Support both connection string and managed identity authentication
- Add comprehensive error handling and Azure-specific exceptions
- Include usage examples and documentation
- Configure as NuGet package with proper metadata
- Integrate seamlessly with existing NebulaStore AFS architecture

This enables NebulaStore to use Azure Blob Storage as a distributed
storage backend, expanding cloud storage options beyond AWS S3.
- Remove macOS .DS_Store file from benchmarks directory
- Create proper test project structure for Azure Storage
- Add comprehensive unit tests for AzureStorageConfiguration
- Add unit tests for AzureStoragePathValidator
- Fix project references and dependencies
- Update main project to exclude test files from compilation
- All 16 tests pass successfully

Tests cover:
- Configuration builder pattern and fluent API
- Path validation for Azure container naming rules
- Factory methods and static constructors
- Default values and parameter validation
@hadv hadv merged commit 5bc00e5 into main Sep 4, 2025
5 checks passed
@hadv hadv deleted the feature/azure-storage-integration branch September 4, 2025 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants