Ash Storage (WIP)

This project continues the design discussion from lifeiscontent/ash_storage#2

What Is This?

An extensible file storage system for Elixir with first-class Ash Framework integration. It provides a consistent interface across disk and cloud backends, supports analysis/transformations/previews, and focuses on strong DB–storage consistency and clear transaction boundaries.

Scope (Initial Milestone)

• Minimal, working local-disk storage.
• Single Ash extension (AshStorage.Attachment) to declaratively attach files to resources.
• Solid transaction semantics (create, attach, purge) with background processing where appropriate.
• Clear seams (behaviours) for adding providers and features incrementally.

Architecture

• Services: pluggable backends (Disk, S3, GCS, Azure, Mirror).
• Core models: Blob (file + metadata), Attachment (record <-> blob), VariantRecord (optional tracking).
• Processing pipeline:
  • Analyzer: extract metadata (images, video, audio).
  • Processor/Transformer: variants (resize, etc.) via ImageMagick/Vips.
  • Previewer: previews for non-images (PDF, video).
• Consistency guarantees:
  • Blob–file, Attachment–blob, Record–attachment, Variant–original.
• Background jobs: analyze, transform, mirror, purge (configurable queues).

Core Operations

• Blob creation (server-side and client direct upload with signed URLs).
• Attachment (single/many) with two-phase commit semantics.
• File analysis (async), representations (variants/previews), and serving (public/signed, redirect/proxy).
• File mirroring across services and purging with orphan cleanup strategies.
• Service configuration at boot with validation.

Ash Integration

• Provide a single extension AshStorage.Attachment for resources to declare attachments, blob domain, storage strategies, and optional transforms—starting simple (local) and expanding to cloud providers.

Roadmap (High Level)

• Local disk adapter (MVP), analyzers for common types, basic variants/previews.
• Signed URL direct uploads, robust purge/mirror flows.
• Cloud services (S3, GCS, Azure).
• Optional open-api–driven codegen for provider SDKs (inspired by open_api_generator/open_api_spex).
• Variant tracking and on-demand generation.

Status

Early design/implementation phase. Expect rapid iteration on APIs and behaviours.