A high-performance, production-ready video/audio streaming service built in Go, designed for 100M+ RPS with HLS adaptive bitrate streaming.
This streaming platform provides enterprise-grade video delivery with:
- HLS Adaptive Streaming - Automatic quality adjustment based on viewer bandwidth
- Multi-Bitrate Encoding - 1080p, 720p, 480p, 360p renditions
- Audio Support - Standalone audio streaming and video audio extraction
- AWS Native - S3 storage, DynamoDB metadata, CloudFront CDN
- Kubernetes Ready - Terraform-managed infrastructure for auto-scaling
┌─────────────────────────────────────────────────────────────────────────────┐
│ CLIENT LAYER │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Web │ │ Mobile │ │ Smart TV │ │ OTT │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
└───────┼─────────────┼─────────────┼─────────────┼───────────────────────────┘
│ │ │ │
└─────────────┴──────┬──────┴─────────────┘
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ CDN / EDGE LAYER │
│ ┌─────────────────┐ │
│ │ CloudFront │ ◄── Cached HLS Segments │
│ └────────┬────────┘ │
└─────────────────────────────────┼───────────────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ API GATEWAY LAYER │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ Load Balancer │──│ API Server │──│ Auth │ │
│ └───────────────┘ └───────┬───────┘ └───────────────┘ │
│ │ │
└─────────────────────────────┼───────────────────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ MICROSERVICES LAYER │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Upload │ │ Transcode │ │ Stream │ │ Audio │ │
│ │ Service │ │ Service │ │ Service │ │ Service │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
└─────────┼────────────────┼────────────────┼────────────────┼────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ DATA & PROCESSING LAYER │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ S3 │ │ FFMPEG │ │ DynamoDB │ │ Redis │ │
│ │ (Storage) │ │ (Encoding) │ │ (Metadata) │ │ (Queue) │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
streaming-service/
├── cmd/
│ ├── api/ # API server entrypoint
│ └── worker/ # Transcoding worker entrypoint
├── internal/
│ ├── api/ # HTTP handlers & Chi router
│ ├── config/ # Viper configuration management
│ ├── domain/ # Business entities (Media, Video, Audio)
│ ├── media/
│ │ ├── ffmpeg/ # FFMPEG video/audio processors
│ │ └── processor/ # Factory & Strategy pattern implementations
│ ├── queue/ # Redis job queue with priority support
│ ├── repository/
│ │ ├── dynamodb/ # Metadata CRUD operations
│ │ └── s3/ # Object storage with presigned URLs
│ └── service/
│ ├── audio/ # Audio extraction & processing
│ ├── stream/ # Playback URL generation
│ ├── transcode/ # HLS transcoding pipeline
│ └── upload/ # File upload handling
├── pkg/
│ └── logger/ # Zap structured logging
├── deployments/
│ ├── docker/ # Multi-stage Dockerfiles
│ ├── kubernetes/ # K8s manifests
│ └── terraform/ # Infrastructure as Code
└── config.yaml # Application configuration
Creates appropriate media processors based on content type:
processor := factory.CreateProcessor(domain.MediaTypeVideo) // or MediaTypeAudioInterchangeable transcoding strategies for different output formats:
executor.AddStrategy(NewHLSTranscodeStrategy(profile1080p))
executor.AddStrategy(NewHLSTranscodeStrategy(profile720p))
executor.Execute(ctx, input, outputDir, cmdExecutor)Concurrent job processing with configurable parallelism:
worker := transcode.NewWorker(queue, service, concurrency, logger)
worker.Start(ctx)- Go 1.22+
- Docker & Docker Compose
- FFMPEG (for local development)
- AWS CLI (configured)
# Clone repository
git clone https://github.com/ichbingautam/streaming-service.git
cd streaming-service
# Install dependencies
go mod download
# Start infrastructure (Redis + LocalStack)
docker-compose up -d redis localstack
# Run API server
make run-api
# Run transcoding worker (in another terminal)
make run-worker# Start all services
docker-compose up -d
# View logs
docker-compose logs -f api worker| Method | Endpoint | Description |
|---|---|---|
GET |
/health |
Health check |
GET |
/ready |
Readiness probe |
POST |
/api/v1/upload |
Upload media file (multipart) |
POST |
/api/v1/upload/presign |
Get presigned upload URL |
POST |
/api/v1/upload/{id}/confirm |
Confirm presigned upload |
GET |
/api/v1/media |
List user's media |
GET |
/api/v1/media/{id} |
Get media details |
DELETE |
/api/v1/media/{id} |
Delete media |
GET |
/api/v1/media/{id}/playback |
Get HLS playback URL |
curl -X POST http://localhost:8080/api/v1/upload \
-H "X-User-ID: user123" \
-F "file=@video.mp4" \
-F "title=My Video" \
-F "description=Sample video"curl http://localhost:8080/api/v1/media/{media_id}/playback \
-H "X-User-ID: user123"Configuration via config.yaml or environment variables (prefix: STREAM_):
app:
name: streaming-service
version: 1.0.0
environment: production
server:
port: 8080
readtimeout: 30s
writetimeout: 30s
aws:
region: us-east-1
s3rawbucket: streaming-raw-media
s3processedbucket: streaming-processed-media
dynamodbtable: video-metadata
cloudfrontdomain: d1234.cloudfront.net
redis:
host: redis
port: 6379
ffmpeg:
binarypath: ffmpeg
segmentduration: 6
profiles:
- name: "1080p"
width: 1920
height: 1080
videobitrate: "5000k"
audiobitrate: "192k"
worker:
concurrency: 4
jobtimeout: 30m
log:
level: info
format: jsoncd deployments/terraform
# Initialize Terraform
terraform init
# Review plan
terraform plan -var="environment=production"
# Apply infrastructure
terraform apply -var="environment=production"# Create namespace
kubectl create namespace streaming
# Apply manifests
kubectl apply -f deployments/kubernetes/ -n streaming
# Check deployment
kubectl get pods -n streaming# Run tests
make test
# Run with coverage
make test-coverage
# Lint code
make lint
# Build binaries
make build
# Build Docker images
make docker-build| Metric | Target |
|---|---|
| API Latency (p99) | < 50ms |
| Transcode Time (1min video) | < 60s |
| CDN Cache Hit Ratio | > 95% |
| Concurrent Streams | 100M+ |
| Component | Technology |
|---|---|
| Language | Go 1.22+ |
| HTTP Router | Chi |
| Configuration | Viper |
| Logging | Zap |
| Media Processing | FFMPEG |
| Storage | AWS S3 |
| Metadata | AWS DynamoDB |
| CDN | AWS CloudFront |
| Queue | Redis |
| Containers | Docker |
| Orchestration | Kubernetes |
| IaC | Terraform |
MIT License - see LICENSE for details.
- Fork the repository
- Create feature branch (
git checkout -b feature/amazing) - Commit changes (
git commit -m 'Add amazing feature') - Push to branch (
git push origin feature/amazing) - Open Pull Request