Distributed Object Storage Gateway

A stateless HTTP gateway for routing object storage requests across MinIO instances using hash-based sharding.

The gateway discovers nodes dynamically via Docker API and routes each object to a consistent node based on its ID. No local state, horizontally scalable.

Quick Start

docker compose up --build

curl -X PUT -d "Hello, World!" http://localhost:3000/object/test123
curl http://localhost:3000/object/test123

How It Works

Client → Gateway (hash(id) % nodes) → MinIO [1|2|3]

Each object ID hashes to the same node every time:

hash := fnv.New32a()
hash.Write([]byte(objectID))
nodeIndex := hash.Sum32() % nodeCount

Same ID → same node. Different IDs → distributed across nodes.

API

PUT /object/{id} - Store object (alphanumeric ID, max 32 chars) GET /object/{id} - Retrieve object

Returns: 200 (success), 404 (not found), 400 (invalid ID), 502 (backend error)

Architecture

├── main.go              # Lifecycle
├── cmd/gateway.go       # Bootstrap
└── internal/
    ├── discovery/       # Docker node discovery
    ├── storage/         # MinIO pool + routing
    └── server/          # HTTP handlers

Modular, testable, production patterns (graceful shutdown, context propagation).

Testing

go test ./...

See TESTING.md for detailed test scenarios.

Tradeoffs

• Modulo hashing - Simple but remaps on node changes (would use consistent hashing for dynamic clusters)
• No replication - Single node per object (would add multi-write with quorum for production)
• Static discovery - Nodes found at startup (would add periodic refresh for auto-scaling)

See IMPLEMENTATION.md for detailed design decisions, production evolution path, and Q&A.

Requirements

Go 1.21+, Docker & Docker Compose

License

MIT - see LICENSE