Skip to content

Cross-platform desktop client for Archivist decentralized storage - built with Tauri v2, React, and Rust

Notifications You must be signed in to change notification settings

durability-labs/archivist-desktop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

77 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Archivist Desktop

A cross-platform desktop application for decentralized file storage, built with Tauri v2, React, and TypeScript.

WARNING: Alpha Software - Pilot Program

This software is in alpha stage and is part of the pilot program. Do not use this for mission-critical data or personal files that you cannot afford to lose.

  • Data loss may occur due to bugs, incomplete features, or network issues
  • There is no guarantee of data persistence or recovery
  • Always maintain separate backups of important files
  • This software is provided "as-is" without warranty of any kind

By using this software, you acknowledge and accept these risks.

Features

  • Guided Onboarding: First-run wizard to get your first backup in under 30 seconds
  • File Management: Upload, download, and manage files on the decentralized network
  • Folder Sync: Watch folders and automatically sync changes to the network
  • Backup Server: Automatic continuous backup to designated peers
  • Peer Network: Connect with peers, share SPR records, and monitor network stats
  • Node Logs: Built-in real-time log viewer with auto-refresh and auto-scroll
  • System Tray: Runs in the background with quick access from the system tray
  • Auto-Update: Automatic updates from GitHub releases
  • Sound Notifications: Audio feedback for node startup, peer connections, and downloads

Tech Stack

Layer Technology
Frontend React 18 + TypeScript + Vite
Backend Rust + Tauri v2
Sidecar archivist-node (P2P storage daemon)
Package Manager pnpm v10
Node.js v20
Rust 1.77.2+ stable

Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚              Archivist Desktop (Tauri App)               β”‚
β”‚                                                          β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚  β”‚  React Frontend    β”‚      β”‚   Rust Backend         β”‚ β”‚
β”‚  β”‚  (Webview)         │◄────►│   (Native Process)     β”‚ β”‚
β”‚  β”‚                    β”‚ IPC  β”‚                        β”‚ β”‚
β”‚  β”‚ β€’ Dashboard        β”‚      β”‚ β€’ Node Management      β”‚ β”‚
β”‚  β”‚ β€’ Backups          β”‚      β”‚ β€’ File Operations      β”‚ β”‚
β”‚  β”‚ β€’ Restore          β”‚      β”‚ β€’ Folder Watching      β”‚ β”‚
β”‚  β”‚ β€’ Devices          β”‚      β”‚ β€’ Peer Management      β”‚ β”‚
β”‚  β”‚ β€’ Peers            β”‚      β”‚ β€’ Backup Daemon        β”‚ β”‚
β”‚  β”‚ β€’ Logs             β”‚      β”‚ β€’ Configuration        β”‚ β”‚
β”‚  β”‚ β€’ Settings         β”‚      β”‚ β€’ HTTP Client          β”‚ β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚                                          β”‚              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                           β”‚
                                  HTTP (localhost:8080)
                                           β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚           archivist-node Sidecar (Separate Process)     β”‚
β”‚                                                          β”‚
β”‚  β€’ REST API (port 8080)                                 β”‚
β”‚  β€’ File Storage & CID Management                        β”‚
β”‚  β€’ P2P Network (libp2p)                                 β”‚
β”‚  β€’ Discovery (DHT/mDNS, UDP port 8090)                  β”‚
β”‚  β€’ Listen (TCP port 8070)                               β”‚
β”‚  β€’ Peer Connections                                     β”‚
β”‚  β€’ Data Replication                                     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                           β”‚
                                   P2P (encrypted)
                                           β”‚
                              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                              β”‚   External Peers        β”‚
                              β”‚   (libp2p network)      β”‚
                              β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

How It Works

  1. User Interface: React frontend provides the UI (Dashboard, Backups, Restore, Devices, Peers, Logs, Settings)
  2. Tauri Backend: Rust backend handles:
    • Starting/stopping the archivist-node sidecar process
    • Managing file system operations (uploads, downloads, folder watching)
    • Proxying requests to the node's REST API
    • Persisting application configuration
    • Running the backup daemon for continuous sync
  3. Archivist Node: Standalone sidecar process that:
    • Exposes REST API on localhost:8080
    • Manages content-addressed storage (CIDs)
    • Handles P2P networking via libp2p
    • Discovers peers via DHT/mDNS on UDP port 8090
    • Accepts peer connections on TCP port 8070
    • Replicates data across the network
  4. P2P Network: Encrypted libp2p connections between peers for file transfer and discovery

Development

Prerequisites

  • Node.js 20+
  • pnpm v10+
  • Rust 1.77.2+ stable
  • Platform-specific dependencies for Tauri (see Tauri Prerequisites)

Setup

# Quick setup (install deps + download sidecar binary)
pnpm setup

# Or step by step:
pnpm install
pnpm download-sidecar  # Downloads archivist-node for your platform

# Run in development mode
pnpm tauri dev

# Build for production
pnpm tauri build

Cross-Platform Builds

The sidecar binary must match your target platform. To download for cross-compilation:

# macOS
bash scripts/download-sidecar.sh x86_64-apple-darwin      # Intel
bash scripts/download-sidecar.sh aarch64-apple-darwin     # Apple Silicon

# Linux
bash scripts/download-sidecar.sh x86_64-unknown-linux-gnu   # x64
bash scripts/download-sidecar.sh aarch64-unknown-linux-gnu  # ARM64

# Windows
bash scripts/download-sidecar.sh x86_64-pc-windows-msvc

Project Structure

archivist-desktop/
β”œβ”€β”€ src/                          # React frontend
β”‚   β”œβ”€β”€ components/               # Reusable UI components
β”‚   β”‚   β”œβ”€β”€ NavAccordion.tsx     # Collapsible navigation sections
β”‚   β”‚   └── NextSteps.tsx        # Post-onboarding guidance
β”‚   β”œβ”€β”€ hooks/                    # Custom React hooks
β”‚   β”‚   β”œβ”€β”€ useNode.ts           # Node lifecycle (start/stop/status)
β”‚   β”‚   β”œβ”€β”€ useSync.ts           # Folder watching + sync queue
β”‚   β”‚   β”œβ”€β”€ usePeers.ts          # Peer connections
β”‚   β”‚   β”œβ”€β”€ useOnboarding.ts     # First-run onboarding state
β”‚   β”‚   β”œβ”€β”€ useSoundNotifications.ts  # Audio feedback
β”‚   β”‚   └── useFeatures.ts       # Feature flag detection
β”‚   β”œβ”€β”€ pages/                    # Route components
β”‚   β”‚   β”œβ”€β”€ Dashboard.tsx        # Main status overview
β”‚   β”‚   β”œβ”€β”€ Onboarding.tsx       # First-run wizard
β”‚   β”‚   β”œβ”€β”€ Files.tsx            # Upload/download/restore files
β”‚   β”‚   β”œβ”€β”€ Sync.tsx             # Watched folder management
β”‚   β”‚   β”œβ”€β”€ Devices.tsx          # Device management
β”‚   β”‚   β”œβ”€β”€ AddDevice.tsx        # Device pairing wizard
β”‚   β”‚   β”œβ”€β”€ Peers.tsx            # P2P network view
β”‚   β”‚   β”œβ”€β”€ BackupServer.tsx     # Backup daemon dashboard
β”‚   β”‚   β”œβ”€β”€ Logs.tsx             # Node logs viewer
β”‚   β”‚   └── Settings.tsx         # App configuration
β”‚   β”œβ”€β”€ lib/                      # Utilities and types
β”‚   β”‚   β”œβ”€β”€ cidValidation.ts     # CID format validation
β”‚   β”‚   └── tauri.ts             # Tauri invoke helpers
β”‚   β”œβ”€β”€ styles/                   # CSS files (terminal aesthetic)
β”‚   β”œβ”€β”€ App.tsx                   # Router + layout
β”‚   └── main.tsx                  # Entry point
β”‚
β”œβ”€β”€ src-tauri/                    # Rust backend
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ main.rs              # App entry (delegates to lib.rs)
β”‚   β”‚   β”œβ”€β”€ lib.rs               # Tauri setup, commands, tray
β”‚   β”‚   β”œβ”€β”€ error.rs             # ArchivistError enum
β”‚   β”‚   β”œβ”€β”€ state.rs             # AppState (service container)
β”‚   β”‚   β”œβ”€β”€ node_api.rs          # HTTP client for sidecar
β”‚   β”‚   β”œβ”€β”€ commands/            # Tauri command handlers
β”‚   β”‚   β”‚   β”œβ”€β”€ node.rs          # start/stop/restart/status/logs
β”‚   β”‚   β”‚   β”œβ”€β”€ files.rs         # upload/download/list/delete
β”‚   β”‚   β”‚   β”œβ”€β”€ sync.rs          # watch folders, sync queue, manifests
β”‚   β”‚   β”‚   β”œβ”€β”€ peers.rs         # connect/disconnect/list
β”‚   β”‚   β”‚   └── system.rs        # config, platform info
β”‚   β”‚   └── services/            # Business logic
β”‚   β”‚       β”œβ”€β”€ node.rs          # Sidecar process management
β”‚   β”‚       β”œβ”€β”€ sync.rs          # File watching (notify crate)
β”‚   β”‚       β”œβ”€β”€ config.rs        # Settings persistence
β”‚   β”‚       β”œβ”€β”€ backup_daemon.rs # Backup daemon (polls source peers)
β”‚   β”‚       └── manifest_server.rs # HTTP manifest discovery server
β”‚   β”œβ”€β”€ resources/               # Bundled assets (video files)
β”‚   β”œβ”€β”€ sidecars/                # archivist-node binaries (gitignored)
β”‚   β”œβ”€β”€ Cargo.toml               # Rust dependencies
β”‚   └── tauri.conf.json          # Tauri configuration
β”‚
β”œβ”€β”€ public/                       # Static assets
β”‚   └── logos/                   # Branding assets
β”‚
β”œβ”€β”€ scripts/
β”‚   └── download-sidecar.sh      # Downloads archivist-node binary
β”‚
β”œβ”€β”€ .github/workflows/
β”‚   β”œβ”€β”€ ci.yml                   # Tests, lint, build checks
β”‚   └── release.yml              # Multi-platform release builds
β”‚
└── package.json                 # npm scripts + dependencies

Configuration

Config File Locations

  • Linux: ~/.config/archivist/config.toml
  • macOS: ~/Library/Application Support/archivist/config.toml
  • Windows: %APPDATA%\archivist\config.toml

Node Configuration

Setting Default Description
data_dir Platform-specific Node data directory
api_port 8080 REST API port
discovery_port 8090 UDP port for DHT/mDNS peer discovery
listen_port 8070 TCP port for P2P connections
max_storage_bytes 10 GB Storage quota
auto_start false Start node on app launch
auto_restart true Restart on failure

Note: Configuration changes require a node restart to take effect.

Backup Server Configuration

Setting Default Description
enabled false Enable backup daemon
poll_interval_secs 30 Check for new manifests every N seconds
max_concurrent_downloads 3 Parallel file downloads
max_retries 3 Retry failed downloads
auto_delete_tombstones true Process file deletions

Network Setup

The application uses multiple ports for P2P networking and backup functionality:

Port Protocol Purpose Required On
8070 TCP P2P connections and file transfers Both machines
8090 UDP Discovery via DHT/mDNS Both machines
8085 TCP Manifest server (backup source) Source machine only
8086 TCP Backup trigger endpoint Backup server only

Minimum required: Open ports 8070 (TCP) and 8090 (UDP) for basic P2P functionality.

For backup system: Also open 8085 on the source machine and 8086 on the backup server.

Linux (UFW)

# Required for P2P
sudo ufw allow 8070/tcp  # P2P connections
sudo ufw allow 8090/udp  # Discovery

# For backup source (Machine A)
sudo ufw allow 8085/tcp  # Manifest server

# For backup server (Machine B)
sudo ufw allow 8086/tcp  # Backup trigger

macOS

The firewall will prompt you to allow connections when the app first runs. Click "Allow" to enable P2P connectivity.

Windows (PowerShell as Administrator)

# Required for P2P
netsh advfirewall firewall add rule name="Archivist P2P" dir=in action=allow protocol=tcp localport=8070
netsh advfirewall firewall add rule name="Archivist Discovery" dir=in action=allow protocol=udp localport=8090

# For backup source (Machine A)
netsh advfirewall firewall add rule name="Archivist Manifest Server" dir=in action=allow protocol=tcp localport=8085

# For backup server (Machine B)
netsh advfirewall firewall add rule name="Archivist Backup Trigger" dir=in action=allow protocol=tcp localport=8086

If you change the ports in Settings β†’ Advanced, update your firewall rules accordingly.

Backup Server Flow

The backup server daemon enables automatic continuous backup from source peers to a designated backup server.

Architecture Overview

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                         MACHINE A (Source Peer)                         β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                                                                         β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                                  β”‚
β”‚  β”‚  Watch Folder    β”‚  1. User adds files                              β”‚
β”‚  β”‚  ~/Documents/    │────────────────┐                                 β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                β”‚                                 β”‚
β”‚           β”‚                           β–Ό                                 β”‚
β”‚           β”‚ 2. File watcher      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    β”‚
β”‚           β”‚    detects changes   β”‚  Sync Service  β”‚                    β”‚
β”‚           └─────────────────────►│  (Desktop App) β”‚                    β”‚
β”‚                                  β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜                    β”‚
β”‚                                           β”‚ 3. Upload files             β”‚
β”‚                                           β”‚    (POST /data)             β”‚
β”‚                                           β–Ό                             β”‚
β”‚                                  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    β”‚
β”‚                                  β”‚ archivist-node β”‚                    β”‚
β”‚                                  β”‚  (Port 8080)   β”‚                    β”‚
β”‚                                  β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜                    β”‚
β”‚                                           β”‚                             β”‚
β”‚                                           β”‚ 4. Store files as CIDs      β”‚
β”‚                                           β”‚    file1.txt β†’ zdj7W...    β”‚
β”‚                                           β”‚    file2.pdf β†’ zDvZR...    β”‚
β”‚                                           β”‚                             β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”        β”‚                             β”‚
β”‚  β”‚ After 10 file changes:       β”‚β—„β”€β”€β”€β”€β”€β”€β”€β”˜ 5. Threshold reached        β”‚
β”‚  β”‚                              β”‚                                       β”‚
β”‚  β”‚ Generate manifest file:      β”‚                                       β”‚
β”‚  β”‚ .archivist-manifest-{id}.jsonβ”‚                                       β”‚
β”‚  β”‚                              β”‚                                       β”‚
β”‚  β”‚ {                            β”‚                                       β”‚
β”‚  β”‚   "source_peer_id": "16Uiu..β”‚                                       β”‚
β”‚  β”‚   "sequence_number": 1,      β”‚                                       β”‚
β”‚  β”‚   "files": [                 β”‚                                       β”‚
β”‚  β”‚     {"path": "file1.txt",    β”‚                                       β”‚
β”‚  β”‚      "cid": "zdj7W..."},     β”‚                                       β”‚
β”‚  β”‚     {"path": "file2.pdf",    β”‚                                       β”‚
β”‚  β”‚      "cid": "zDvZR..."}      β”‚                                       β”‚
β”‚  β”‚   ]                          β”‚                                       β”‚
β”‚  β”‚ }                            β”‚                                       β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                       β”‚
β”‚                 β”‚                                                       β”‚
β”‚                 β”‚ 6. Upload manifest                                    β”‚
β”‚                 β”‚    (POST /data)                                       β”‚
β”‚                 β–Ό                                                       β”‚
β”‚        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                              β”‚
β”‚        β”‚ archivist-node β”‚                                              β”‚
β”‚        β”‚  Manifest CID: β”‚                                              β”‚
β”‚        β”‚  zDvZRwzm...   β”‚                                              β”‚
β”‚        β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜                                              β”‚
β”‚                 β”‚                                                       β”‚
β”‚                 β”‚ 7. Create storage request                            β”‚
β”‚                 β”‚    for backup peer                                   β”‚
β”‚                 β”‚                                                       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                  β”‚
                  β”‚ 8. P2P Network
                  β”‚    (libp2p encrypted)
                  β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                        MACHINE B (Backup Server)                        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                                                                         β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                β”‚
β”‚  β”‚      Backup Daemon (Background)    β”‚                                β”‚
β”‚  β”‚   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”‚                                β”‚
β”‚  β”‚   β”‚ Every 30 seconds:          β”‚   β”‚  9. Poll for manifests         β”‚
β”‚  β”‚   β”‚ GET /data                  │───┼─────────────┐                  β”‚
β”‚  β”‚   β”‚ Filter: *.manifest*.json   β”‚   β”‚             β”‚                  β”‚
β”‚  β”‚   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚             β”‚                  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜             β”‚                  β”‚
β”‚                   β”‚                                  β–Ό                  β”‚
β”‚                   β”‚ 10. Manifest     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”         β”‚
β”‚                   β”‚     discovered   β”‚   archivist-node      β”‚         β”‚
β”‚                   β”‚                  β”‚   (Port 8080)         β”‚         β”‚
β”‚                   β”‚                  β”‚                       β”‚         β”‚
β”‚                   β”‚                  β”‚ Files stored:         β”‚         β”‚
β”‚                   β”‚                  β”‚ β€’ manifest.json       β”‚         β”‚
β”‚                   β”‚                  β”‚ β€’ file1.txt (zdj7W)   β”‚         β”‚
β”‚                   β”‚                  β”‚ β€’ file2.pdf (zDvZR)   β”‚         β”‚
β”‚                   β”‚                  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜         β”‚
β”‚                   β”‚                                                     β”‚
β”‚                   β”‚ 11. Parse manifest                                 β”‚
β”‚                   β”‚     Extract CID list                               β”‚
β”‚                   β”‚                                                     β”‚
β”‚                   β–Ό                                                     β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                    β”‚
β”‚  β”‚  Download missing files        β”‚  12. For each CID:                 β”‚
β”‚  β”‚  (3 concurrent downloads)      β”‚      POST /data/{cid}/network      β”‚
β”‚  β”‚                                β”‚                                    β”‚
β”‚  β”‚  zdj7W... β–“β–“β–“β–“β–“β–“β–“β–“β–‘β–‘ 80%      β”‚      (Download from network        β”‚
β”‚  β”‚  zDvZR... β–“β–“β–“β–“β–“β–“β–“β–“β–“β–“ 100%     β”‚       via P2P from Machine A)      β”‚
β”‚  β”‚  zDpuA... β–“β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘ 10%      β”‚                                    β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                    β”‚
β”‚                   β”‚                                                     β”‚
β”‚                   β”‚ 13. Update state                                   β”‚
β”‚                   β–Ό                                                     β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                    β”‚
β”‚  β”‚  backup-daemon-state.json      β”‚                                    β”‚
β”‚  β”‚                                β”‚                                    β”‚
β”‚  β”‚  {                             β”‚                                    β”‚
β”‚  β”‚    "processed_manifests": {    β”‚                                    β”‚
β”‚  β”‚      "zDvZRwzm...": {          β”‚                                    β”‚
β”‚  β”‚        "source_peer_id": "...", β”‚                                    β”‚
β”‚  β”‚        "sequence_number": 1,   β”‚                                    β”‚
β”‚  β”‚        "file_count": 15,       β”‚                                    β”‚
β”‚  β”‚        "total_size_bytes": ... β”‚                                    β”‚
β”‚  β”‚      }                          β”‚                                    β”‚
β”‚  β”‚    },                           β”‚                                    β”‚
β”‚  β”‚    "stats": {                   β”‚                                    β”‚
β”‚  β”‚      "total_manifests": 1,     β”‚                                    β”‚
β”‚  β”‚      "total_files": 15,        β”‚                                    β”‚
β”‚  β”‚      "total_bytes": ...        β”‚                                    β”‚
β”‚  β”‚    }                            β”‚                                    β”‚
β”‚  β”‚  }                             β”‚                                    β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                    β”‚
β”‚                                                                         β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                    β”‚
β”‚  β”‚  Backup Server Dashboard       β”‚  14. User views status             β”‚
β”‚  β”‚  (http://localhost:1420)       β”‚                                    β”‚
β”‚  β”‚                                β”‚                                    β”‚
β”‚  β”‚  Manifests Processed: 1        β”‚                                    β”‚
β”‚  β”‚  Files Downloaded: 15          β”‚                                    β”‚
β”‚  β”‚  Total Size: 2.4 MB            β”‚                                    β”‚
β”‚  β”‚                                β”‚                                    β”‚
β”‚  β”‚  Processed Manifests           β”‚                                    β”‚
β”‚  β”‚  Source: 16Uiu2HAm... (Seq #1) β”‚                                    β”‚
β”‚  β”‚  Files: 15 | Size: 2.4 MB      β”‚                                    β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                    β”‚
β”‚                                                                         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Step-by-Step Process

Machine A (Source Peer):

  1. User adds files to watched folder
  2. File watcher detects changes (create/modify/delete)
  3. Sync service uploads files to local archivist-node via POST /data
  4. Node stores files and returns CIDs (content identifiers)
  5. After 10 file changes (configurable threshold), manifest is generated
  6. Manifest file created: .archivist-manifest-{peer_id}.json containing:
    • Source peer ID
    • Sequence number (increments with each update)
    • List of all files with their CIDs
    • Deleted files (tombstones for cleanup)
  7. Manifest uploaded to local node, gets its own CID
  8. Storage request created for backup peer (if configured)

P2P Network:

  • Manifest propagates through libp2p network
  • Peers exchange data using encrypted connections
  • Content-addressed storage ensures data integrity

Machine B (Backup Server):

  1. Backup daemon polls /data endpoint every 30 seconds
  2. Discovers new manifest files (filter: *.manifest*.json)
  3. Downloads and parses manifest to extract CID list
  4. For each CID in manifest:
    • Check if already stored locally
    • If missing: POST /data/{cid}/network to download from network
    • Downloads happen concurrently (3 at a time by default)
  5. Updates daemon state file with:
    • Processed manifests
    • Statistics (files downloaded, bytes, etc.)
    • Failed downloads (for retry)
  6. Dashboard displays real-time backup status

Source Peer Configuration

# In config.toml on Machine A
[sync]
backup_enabled = true
backup_peer_address = "spr:CiUIAhIhAml6..."  # Machine B's SPR
backup_manifest_enabled = true
backup_auto_notify = true
manifest_update_threshold = 10  # Generate manifest after N file changes

Backup Server Configuration

# In config.toml on Machine B
[backup_server]
enabled = true
poll_interval_secs = 30          # Check for new manifests every 30s
max_concurrent_downloads = 3     # Download 3 files at once
max_retries = 3                  # Retry failed downloads 3 times
auto_delete_tombstones = true    # Process file deletions

Key Features

Feature Description
Event-Driven Manifests generated automatically after threshold reached
Continuous Sync New files trigger manifest updates without manual intervention
Deletion Tracking Deleted files tracked in manifest for proper cleanup
Sequence Numbers Detect gaps and ensure proper ordering
Retry Mechanism Failed downloads automatically retried with backoff
Concurrent Downloads Multiple files downloaded in parallel for speed
State Persistence Daemon state saved to disk, survives restarts
Real-Time Dashboard Monitor backup progress with auto-refreshing UI
n:1 Fan-In Multiple source peers can backup to single server
Content Deduplication Same file content = same CID = stored once

Network Requirements

For cross-network backup (Machine A β†’ Internet β†’ Machine B):

  1. Machine A (Source) port forwarding:

    • Forward external port 8070 (TCP) β†’ Machine A's local IP:8070 (P2P)
    • Forward external port 8085 (TCP) β†’ Machine A's local IP:8085 (Manifest server)
  2. Machine B (Backup Server) port forwarding:

    • Forward external port 8070 (TCP) β†’ Machine B's local IP:8070 (P2P)
    • Forward external port 8086 (TCP) β†’ Machine B's local IP:8086 (Backup trigger, optional)
  3. Firewall rules:

    • Machine A: Allow TCP 8070, 8085 and UDP 8090
    • Machine B: Allow TCP 8070, 8086 and UDP 8090
  4. Connection tip: If NAT traversal fails in one direction, try connecting from the other machine first. The machine with more permissive NAT/firewall should initiate the P2P connection.

Troubleshooting

Common Issues

Issue Solution
Port 8080 in use Change API port in Settings β†’ Advanced
Sidecar not found Run pnpm download-sidecar
0 addresses found Check firewall allows ports 8090 (UDP) and 8070 (TCP)
Peer connects then disconnects Check NAT timeout, try reconnecting with fresh SPR

Logs

Node logs are written to:

  • Linux: ~/.local/share/archivist/node.log
  • macOS: ~/Library/Application Support/archivist/node.log
  • Windows: %APPDATA%\archivist\node.log

Use the built-in Logs page for real-time viewing with auto-refresh.

Resources

License

MIT


This software is provided for evaluation and testing purposes as part of the pilot program. See the warning at the top of this document regarding data safety.

About

Cross-platform desktop client for Archivist decentralized storage - built with Tauri v2, React, and Rust

Resources

Stars

Watchers

Forks

Packages

No packages published

Contributors 2

  •  
  •