Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
352 changes: 218 additions & 134 deletions NebulaStore.sln

Large diffs are not rendered by default.

34 changes: 34 additions & 0 deletions afs/nio/NebulaStore.Afs.Nio.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net9.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<LangVersion>latest</LangVersion>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<WarningsAsErrors />
<WarningsNotAsErrors>CS1591</WarningsNotAsErrors>
</PropertyGroup>

<PropertyGroup>
<AssemblyTitle>NebulaStore Abstract File System - NIO</AssemblyTitle>
<AssemblyDescription>.NET I/O Adapter for the NebulaStore Abstract File System</AssemblyDescription>
<AssemblyCompany>NebulaStore</AssemblyCompany>
<AssemblyProduct>NebulaStore</AssemblyProduct>
<Copyright>Copyright © NebulaStore 2025</Copyright>
<AssemblyVersion>1.0.0.0</AssemblyVersion>
<FileVersion>1.0.0.0</FileVersion>
<RootNamespace>NebulaStore.Afs.Nio</RootNamespace>
</PropertyGroup>

<ItemGroup>
<Compile Remove="test/**/*.cs" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="../blobstore/NebulaStore.Afs.Blobstore.csproj" />
</ItemGroup>

</Project>

241 changes: 241 additions & 0 deletions afs/nio/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
# NebulaStore Abstract File System - NIO

The NIO (New I/O) module provides a .NET I/O adapter for the NebulaStore Abstract File System. This implementation uses standard .NET file system APIs (`System.IO`) to provide file operations, making it the default and most straightforward storage backend for local file systems.

## Overview

The NIO module is a direct port of the Eclipse Store AFS NIO module from Java to .NET. It provides:

- **Local File System Access**: Direct access to the local file system using .NET's `System.IO`
- **File Stream Management**: Efficient file stream handling with automatic positioning
- **Path Resolution**: Flexible path resolution with home directory expansion support
- **Thread-Safe Operations**: All operations are thread-safe with proper synchronization
- **Integration**: Seamless integration with the NebulaStore AFS infrastructure

## Architecture

### Core Components

1. **NioFileSystem**: Main entry point for NIO file system operations
2. **NioIoHandler**: Handles all I/O operations (read, write, copy, move, etc.)
3. **NioConnector**: Implements `IBlobStoreConnector` for AFS integration
4. **NioReadableFile**: Wrapper for read-only file access
5. **NioWritableFile**: Wrapper for read-write file access
6. **NioPathResolver**: Resolves path elements to file system paths

### Class Hierarchy

```
INioFileSystem
└── NioFileSystem

INioIoHandler
└── NioIoHandler

INioFileWrapper
├── INioReadableFile
│ └── NioReadableFile<TUser>
└── INioWritableFile
└── NioWritableFile<TUser>

IBlobStoreConnector
└── NioConnector
```

## Usage

### Basic File System Operations

```csharp
using NebulaStore.Afs.Nio;
using NebulaStore.Afs.Blobstore;

// Create a NIO file system
using var fileSystem = NioFileSystem.New();

// Create a path
var path = BlobStorePath.New("container", "folder", "file.txt");

// Wrap for writing
using var writableFile = fileSystem.WrapForWriting(path);

// Write data
var data = System.Text.Encoding.UTF8.GetBytes("Hello, NIO!");
fileSystem.IoHandler.WriteBytes(writableFile, data);

// Wrap for reading
using var readableFile = fileSystem.WrapForReading(path);

// Read data
var readData = fileSystem.IoHandler.ReadBytes(readableFile);
var content = System.Text.Encoding.UTF8.GetString(readData);
Console.WriteLine(content); // Output: Hello, NIO!
```

### Using NioConnector with BlobStoreFileSystem

```csharp
using NebulaStore.Afs.Nio;
using NebulaStore.Afs.Blobstore;

// Create a NIO connector
using var connector = NioConnector.New("./my-storage", useCache: false);

// Create a blob store file system with NIO backend
using var fileSystem = BlobStoreFileSystem.New(connector);

// Use the file system
var path = BlobStorePath.New("data", "users", "user1.dat");
var userData = System.Text.Encoding.UTF8.GetBytes("User data");

fileSystem.IoHandler.WriteData(path, userData);
var retrievedData = fileSystem.IoHandler.ReadData(path, 0, -1);
```

### Integration with EmbeddedStorage

```csharp
using NebulaStore.Storage.Embedded;
using NebulaStore.Storage.EmbeddedConfiguration;

// Configure storage to use NIO backend
var config = EmbeddedStorageConfiguration.New()
.SetStorageDirectory("./nio-storage")
.SetUseAfs(true)
.SetAfsStorageType("nio") // Use NIO backend
.Build();

using var storage = EmbeddedStorage.Start(config);

// Use storage normally
var root = storage.Root<MyDataClass>();
root.SomeProperty = "value";
storage.StoreRoot();
```

### Advanced File Operations

```csharp
using NebulaStore.Afs.Nio;
using NebulaStore.Afs.Blobstore;

using var fileSystem = NioFileSystem.New();
var ioHandler = fileSystem.IoHandler;

// Create directory
var dirPath = BlobStorePath.New("container", "subfolder");
ioHandler.CreateDirectory(dirPath);

// List files
var filePath = BlobStorePath.New("container");
var files = ioHandler.ListFiles(filePath);
foreach (var file in files)
{
Console.WriteLine($"File: {file}");
}

// Copy file
var sourcePath = BlobStorePath.New("container", "source.txt");
var targetPath = BlobStorePath.New("container", "target.txt");

using var sourceFile = fileSystem.WrapForReading(sourcePath);
using var targetFile = fileSystem.WrapForWriting(targetPath);

ioHandler.CopyFile(sourceFile, targetFile);

// Move file
var newPath = BlobStorePath.New("container", "moved.txt");
using var movableFile = fileSystem.WrapForWriting(targetPath);
using var destinationFile = fileSystem.WrapForWriting(newPath);

ioHandler.MoveFile(movableFile, destinationFile);
```

### Path Resolution

```csharp
using NebulaStore.Afs.Nio;

var fileSystem = NioFileSystem.New();

// Home directory expansion
var pathElements = fileSystem.ResolvePath("~/Documents/data");
// Returns: ["Users", "username", "Documents", "data"] (on Unix-like systems)

// Regular path
var elements = fileSystem.ResolvePath("/var/data/files");
// Returns: ["var", "data", "files"]
```

## Features

### File Stream Management

- **Automatic Positioning**: File streams are automatically positioned at the end for append operations
- **Stream Lifecycle**: Proper stream opening, closing, and disposal
- **Reopen Support**: Ability to reopen streams with different options
- **Validation**: Validates file access modes and options

### Thread Safety

All operations are thread-safe:
- File wrapper operations use internal mutex for synchronization
- I/O handler operations are stateless and thread-safe
- File system operations properly handle concurrent access

### Error Handling

Comprehensive error handling:
- `FileNotFoundException` for missing files
- `InvalidOperationException` for retired file wrappers
- `ArgumentException` for invalid parameters
- `IOException` for I/O errors

## Comparison with Other AFS Backends

| Feature | NIO | BlobStore | AWS S3 | Azure | Redis |
|---------|-----|-----------|--------|-------|-------|
| Local Files | ✓ | ✓ | ✗ | ✗ | ✗ |
| Cloud Storage | ✗ | ✗ | ✓ | ✓ | ✗ |
| In-Memory | ✗ | ✗ | ✗ | ✗ | ✓ |
| Caching | ✗ | ✓ | ✓ | ✓ | N/A |
| Performance | High | High | Medium | Medium | Very High |
| Setup Complexity | Low | Low | Medium | Medium | Low |

## Performance Considerations

- **Direct I/O**: Uses .NET's native file I/O for maximum performance
- **No Overhead**: Minimal abstraction overhead compared to other backends
- **Buffering**: Leverages .NET's built-in buffering mechanisms
- **Streaming**: Supports efficient streaming for large files

## Best Practices

1. **Dispose Resources**: Always dispose file wrappers and file systems
2. **Use Using Statements**: Leverage C#'s `using` statement for automatic disposal
3. **Handle Exceptions**: Properly handle file I/O exceptions
4. **Path Validation**: Validate paths before operations
5. **Concurrent Access**: Be aware of file locking when multiple processes access the same files

## Limitations

- **Local Only**: Only supports local file system access
- **No Caching**: Does not include built-in caching (use BlobStore with NioConnector for caching)
- **Platform-Specific**: Path handling may vary across platforms (Windows vs. Unix-like)

## Migration from Java Eclipse Store

This module is a direct port of the Eclipse Store AFS NIO module. Key differences:

- `FileChannel` → `FileStream`
- `Path` (Java NIO) → `string` (file system path)
- `OpenOption` → `FileMode`, `FileAccess`, `FileShare`
- `ByteBuffer` → `byte[]`
- Exception types adapted to .NET conventions

## See Also

- [AFS Overview](../README.md)
- [BlobStore Module](../blobstore/README.md)
- [Eclipse Store NIO Documentation](https://github.com/eclipse-store/store/tree/main/afs/nio)

88 changes: 88 additions & 0 deletions afs/nio/src/INioFileWrapper.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
using System;
using System.IO;

namespace NebulaStore.Afs.Nio;

/// <summary>
/// Interface for NIO file wrappers that provide file channel access.
/// </summary>
public interface INioFileWrapper : INioItemWrapper, IDisposable
{
/// <summary>
/// Gets the file stream (equivalent to Java's FileChannel).
/// </summary>
FileStream? FileStream { get; }

/// <summary>
/// Gets the user context associated with this file.
/// </summary>
object? User { get; }

/// <summary>
/// Retires this file wrapper, preventing further operations.
/// </summary>
/// <returns>True if the file was retired, false if already retired</returns>
bool Retire();

/// <summary>
/// Gets a value indicating whether this file wrapper is retired.
/// </summary>
bool IsRetired { get; }

/// <summary>
/// Gets a value indicating whether the file stream is open.
/// </summary>
bool IsStreamOpen { get; }

/// <summary>
/// Checks if the file stream is open and validates the file is not retired.
/// </summary>
/// <returns>True if the stream is open</returns>
bool CheckStreamOpen();

/// <summary>
/// Ensures the file stream is open, opening it if necessary.
/// </summary>
/// <returns>The open file stream</returns>
FileStream EnsureOpenStream();

/// <summary>
/// Ensures the file stream is open with specific options.
/// </summary>
/// <param name="mode">The file mode</param>
/// <param name="access">The file access</param>
/// <param name="share">The file share mode</param>
/// <returns>The open file stream</returns>
FileStream EnsureOpenStream(FileMode mode, FileAccess access, FileShare share);

/// <summary>
/// Opens the file stream.
/// </summary>
/// <returns>True if the stream was opened, false if already open</returns>
bool OpenStream();

/// <summary>
/// Opens the file stream with specific options.
/// </summary>
/// <param name="mode">The file mode</param>
/// <param name="access">The file access</param>
/// <param name="share">The file share mode</param>
/// <returns>True if the stream was opened, false if already open</returns>
bool OpenStream(FileMode mode, FileAccess access, FileShare share);

/// <summary>
/// Reopens the file stream with specific options.
/// </summary>
/// <param name="mode">The file mode</param>
/// <param name="access">The file access</param>
/// <param name="share">The file share mode</param>
/// <returns>True if the stream was reopened</returns>
bool ReopenStream(FileMode mode, FileAccess access, FileShare share);

/// <summary>
/// Closes the file stream.
/// </summary>
/// <returns>True if the stream was closed, false if already closed</returns>
bool CloseStream();
}

20 changes: 20 additions & 0 deletions afs/nio/src/INioItemWrapper.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
using NebulaStore.Afs.Blobstore;

namespace NebulaStore.Afs.Nio;

/// <summary>
/// Interface for NIO item wrappers that provide access to file system paths.
/// </summary>
public interface INioItemWrapper
{
/// <summary>
/// Gets the file system path for this item.
/// </summary>
string Path { get; }

/// <summary>
/// Gets the blob store path representation.
/// </summary>
BlobStorePath BlobPath { get; }
}

Loading
Loading