Peer-to-peer file sharing and distributed storage using PeerPigeon
PigeonFS enables direct file transfer between browsers with no central server. Send any file type (videos, images, documents, etc.) directly to peers, plus distributed key-value storage and searchable datasets.
- π Any File Type - Send videos (MP4), images, PDFs, documents, archives - anything!
- π Direct Transfer - Files sent peer-to-peer via WebRTC data channels (no server middleman)
- β‘ Chunked Streaming - 64KB chunks with progress tracking for large files
- π Automatic Reconnect - Handles connection drops and peer discovery
- πΎ Browser Download - Received files can be saved directly to disk
- π Transfer Stats - Real-time progress, speed, and file info
- π Optional Encryption - End-to-end encryption with AES-GCM (toggle in UI)
- 𧩠Intelligent Chunking - Automatic file splitting into 64KB chunks
- π Platform-Aware - Different strategies for browser vs node peers
- Browser: Store only chunks based on DHT proximity (saves space)
- Node/App: Use 10% of quota for chunks + whole files if they fit
- οΏ½ Filename Hash Index - Fast O(1) lookups with case-insensitive search
- π Quota Management - Automatic LRU eviction and cleanup
- π DHT Distribution - Chunks distributed via consistent hashing
- πΎ Hybrid Storage - Small files stored whole, large files chunked
- π JSON Dataset Upload - Upload structured datasets to make searchable locally
- π P2P Search - Search datasets without loading them - queries route through peers
- π Multiple Datasets - Generic system supports any dataset (Bible, dictionaries, wikis, etc.)
- β‘ Book.js Integration - Fast radix tree indexing for efficient full-text search
- πΎ Smart Caching - Auto-caches search results to IndexedDB to contribute to network robustness
- π Gossip Protocol - Search requests broadcast to all connected peers
- π Real-time Stats - Track local/peer results, network latency, and storage usage
- PeerPigeon - P2P networking, gossip protocol, peer discovery, WebRTC data channels
- WebRTC Data Channels - Direct browser-to-browser file streaming
- PagingStorage - Distributed hash table for key-value storage and sync
- Book.js - Radix tree structure for fast prefix/substring search on datasets
- Vue 3 - Reactive UI with real-time updates
- IndexedDB - Persistent storage for DHT data, datasets, and cached results
- Vite - Fast dev server and build system
npm install pigeonfsgit clone https://github.com/PeerPigeon/PigeonFS.git
cd PigeonFS
npm installDownload and run the desktop application with integrated server management:
# Run in Electron (development)
npm run electron
# Build desktop app for distribution
npm run electron:buildFeatures:
- π₯οΈ Native desktop app for Windows, macOS, and Linux
- ποΈ Built-in server control panel with GUI
- π Live server logs and status monitoring
- βοΈ Easy configuration without terminal commands
See ELECTRON.md for building and distribution.
npm run devnpm run build
npm run preview- Chunk Storage: see
docs/CHUNK_STORAGE.mdanddocs/CHUNK_STORAGE_QUICK_REF.md
Run a persistent Node.js peer to host files and datasets. Includes web upload interface at http://localhost:3000.
# Run with Node.js
npm run server
# Or build standalone executable (no Node.js required)
npm run package
./dist/pigeonfs-server-macosUpload files via:
- Browser UI - Built-in upload interface in the "Server File Management" section
- Web interface - Drag & drop at
http://localhost:3000 - HTTP API -
curl -F "file=@video.mp4" http://localhost:3000/upload - Manual placement - Copy to
data/files/directory
Browser features:
- π€ Upload files directly to server from browser
- π Search server files using Book.js index
- β¬οΈ Download files from server
- π View complete file list
See NODE_SERVER.md for full details.
Open in multiple browser windows/tabs to test P2P file transfer and storage across peers.
- Connect to network - Enter a network namespace and click Connect
- Select a file - Choose ANY file (video, image, PDF, etc.) from your computer
- Get peer ID - Copy your peer ID and share with recipient
- Send file - Paste recipient's peer ID and click "Send File"
- Receive file - File appears in recipient's "Received Files" section
- Download - Click download button to save file to disk
Works with:
- π₯ Videos (MP4, WebM, AVI, etc.)
- πΌοΈ Images (JPG, PNG, GIF, etc.)
- π Documents (PDF, DOCX, TXT, etc.)
- π¦ Archives (ZIP, RAR, TAR, etc.)
- οΏ½οΏ½ Audio (MP3, WAV, OGG, etc.)
- Any other file type!
- Connect to network - Enter a network namespace and click Connect
- Store key-value data - Use the PagingStorage section:
// Automatically routes to responsible peers via DHT
await storage.set('user:123', { name: 'Alice', age: 30 })
await storage.set('file:readme', 'Hello World!')- Retrieve from any peer - Data automatically fetched from DHT:
const user = await storage.get('user:123') // Routes to correct peer- View stats - See pages, keys, replication status across network
- Export/Import - Download or restore full storage state
- Load a dataset - Click "Load King James Bible" (or add your own dataset)
- Search locally - Type query like "faith" and press Enter
- Search via peers - Open another tab/window, DON'T load the dataset, just search
- See peer results - First tab serves results to second tab via P2P network
Enable end-to-end encryption for all P2P communications:
Browser:
- Check the "π Enable End-to-End Encryption" checkbox before connecting
- All messages, file transfers, and storage will be encrypted
Node.js Server:
ENABLE_CRYPTO=true npm run serverHow it works:
- Uses UnSEA crypto library (AES-GCM + ECDH key exchange)
- Each peer generates ECDSA key pair on initialization
- Messages encrypted with recipient's public key
- ~10-20% performance overhead
β οΈ All peers must use the same encryption setting (all on or all off)
When to use encryption:
- Sensitive file transfers (medical records, legal documents, etc.)
- Private conversations or data
- Untrusted networks
- Compliance requirements (HIPAA, GDPR, etc.)
- Auto-caching - Second tab caches received results to help other peers
Sender PeerPigeon Network Receiver
| | |
|--select file---------------->| |
|--get peer ID---------------->| |
| |<--share peer ID---------|
| | |
|--establish WebRTC----------->|<--establish WebRTC------|
| | |
|==file chunks (64KB)=========>|==file chunks==========>|
| | |--save to disk
Client DHT Hash Ring Replica Peers (3x)
| | |
|--set('key', value)------>| |
| |--hash(key)--------------->|
| |--route to peers---------->|
| | |--store locally
| |<--acknowledge-------------|
|<--success----------------| |
Searcher Network Data Holder
| | |
|---(search query)-------->| |
| |---(gossip broadcast)-------->|
| | |---(index search)
| | |
| |<---(results)-----------------|
|<---(results + cache)-----| |
Currently includes:
- King James Bible - 31,102 verses, full-text searchable
Easily add more datasets by adding to availableDatasets object with:
name- Display nameicon- Emoji iconurl- Dataset JSON URLloader- Transform function to convert to Book.js format
PigeonFS uses Book.js for its searchable dataset feature, which was inspired by Gun DB and Mark Nadal's vision for decentralized data structures. The Book.js radix tree implementation enables fast full-text search across distributed datasets.
Special thanks to:
- Mark Nadal - For pioneering P2P database concepts and Gun DB, which inspired the Book.js search architecture
- Gun DB community - For inspiration on distributed data structures
- Book.js - Fast radix tree implementation for full-text search
- PeerPigeon - WebRTC mesh networking and gossip protocol
MIT