Skip to content

TestCase-Titans/cuet-micro-ops-hackthon-2025

BranchesTags

Repository files navigation

CUET Micro-Ops Hackathon 2025

CI Status License: MIT

Delineate Hackathon Challenge - CUET Fest 2025

A microservice simulating real-world file download systems with variable processing times, designed to explore solutions for long-running operations behind reverse proxies.

The Challenge

This microservice handles downloads with dramatically different processing times:

┌─────────────────────────────────────────────────────────────────────────┐
│ Download Processing Time                                                │
├─────────────────────────────────────────────────────────────────────────┤
│ Fast Downloads     ████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ~10-15s   │
│ Medium Downloads   ████████████████████░░░░░░░░░░░░░░░░░░░░ ~30-60s   │
│ Slow Downloads     ████████████████████████████████████████ ~60-120s  │
└─────────────────────────────────────────────────────────────────────────┘

Why This Matters

When deployed behind reverse proxies (Cloudflare, nginx, AWS ALB), you'll face:

  • Connection Timeouts - Cloudflare's 100s timeout kills long requests
  • Gateway Errors - Users see 504 errors for slow downloads
  • Poor UX - No progress feedback during long waits
  • Resource Waste - Open connections consume server memory

Hackathon Challenges

Challenge Max Points Difficulty
Challenge 1: S3 Storage Integration 15 Medium
Challenge 2: Architecture Design 15 Hard
Challenge 3: CI/CD Pipeline 10 Medium
Challenge 4: Observability (Bonus) 10 Hard
Maximum Total 50

Technical Requirements

Requirement Version
Node.js >= 24.10.0
npm >= 10.x
Docker >= 24.x
Docker Compose >= 2.x

Tech Stack

  • Runtime: Node.js 24 with native TypeScript support
  • Framework: Hono - Ultra-fast web framework
  • Validation: Zod with OpenAPI integration
  • Storage: AWS S3 SDK (S3-compatible)
  • Observability: OpenTelemetry + Jaeger
  • Error Tracking: Sentry
  • Documentation: Scalar OpenAPI UI

Quick Start

Prerequisites

Ensure you have the required versions installed:

node --version  # Should be >= 24.10.0
npm --version   # Should be >= 10.x
docker --version # Should be >= 24.x

Local Development

# Clone the repository
git clone https://github.com/bongodev/cuet-micro-ops-hackthon-2025.git
cd cuet-micro-ops-hackthon-2025

# Install dependencies
npm install

# Create environment file
cp .env.example .env

# Start development server (with hot reload, 5-15s delays)
npm run dev

# Or start production server (10-120s delays)
npm run start

The server will start at http://localhost:3000

Using Docker

# Development mode (with Jaeger tracing)
npm run docker:dev

# Production mode
npm run docker:prod

# Stop services
npm run docker:down

Services Available:

Environment Variables

Create a .env file in the project root:

# Server Configuration
NODE_ENV=development
PORT=3000

# S3 Configuration
S3_REGION=us-east-1
S3_ENDPOINT=http://localhost:9000
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_BUCKET_NAME=downloads
S3_FORCE_PATH_STYLE=true

# Observability (Optional)
SENTRY_DSN=
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

# Rate Limiting
REQUEST_TIMEOUT_MS=30000
RATE_LIMIT_WINDOW_MS=60000
RATE_LIMIT_MAX_REQUESTS=100

# CORS
CORS_ORIGINS=*

# Download Delay Simulation
DOWNLOAD_DELAY_ENABLED=true
DOWNLOAD_DELAY_MIN_MS=10000
DOWNLOAD_DELAY_MAX_MS=120000

API Endpoints

Method Endpoint Description
GET / Welcome message
GET /health Health check with storage status
GET /docs Interactive API documentation
GET /openapi OpenAPI specification
POST /v1/download/initiate Initiate bulk download job
POST /v1/download/check Check single file availability
POST /v1/download/start Start download with simulated delay

Testing the Long-Running Download

# With dev server (5-15s delays)
npm run dev

curl -X POST http://localhost:3000/v1/download/start \
  -H "Content-Type: application/json" \
  -d '{"file_id": 70000}'

# With production server (10-120s delays) - may timeout!
npm run start

curl -X POST http://localhost:3000/v1/download/start \
  -H "Content-Type: application/json" \
  -d '{"file_id": 70000}'

Available Scripts

Command Description
npm run dev Start dev server (5-15s delays, hot reload)
npm run start Start production server (10-120s delays)
npm run lint Run ESLint
npm run lint:fix Fix linting issues automatically
npm run format Format code with Prettier
npm run format:check Check code formatting
npm run test:e2e Run E2E tests
npm run docker:dev Start with Docker (development)
npm run docker:prod Start with Docker (production)
npm run docker:down Stop Docker services

Project Structure

.
├── src/
│   └── index.ts              # Main application entry point
├── scripts/
│   ├── e2e-test.ts           # E2E test suite
│   └── run-e2e.ts            # Test runner with server management
├── docker/
│   ├── Dockerfile.dev        # Development Dockerfile
│   ├── Dockerfile.prod       # Production Dockerfile
│   ├── compose.dev.yml       # Development Docker Compose
│   └── compose.prod.yml      # Production Docker Compose
├── .github/
│   └── workflows/
│       └── ci.yml            # GitHub Actions CI pipeline
├── package.json
├── tsconfig.json
├── eslint.config.mjs
├── .env.example
└── README.md

Security Features

  • Request ID tracking for distributed tracing
  • Rate limiting with configurable windows
  • Security headers (HSTS, X-Frame-Options, etc.)
  • CORS configuration
  • Input validation with Zod schemas
  • Path traversal prevention for S3 keys
  • Graceful shutdown handling

CI/CD Pipeline

This project uses GitHub Actions for continuous integration and deployment.

Pipeline Status

CI Status

Pipeline Stages

┌─────────────┐   ┌─────────────┐   ┌─────────────┐   ┌─────────────┐
│    Lint     │──▶│    Test     │──▶│    Build    │──▶│   Deploy    │
│  (ESLint,   │   │    (E2E)    │   │  (Docker)   │   │ (Optional)  │
│  Prettier)  │   │             │   │             │   │             │
└─────────────┘   └─────────────┘   └─────────────┘   └─────────────┘

Running Tests Locally Before Pushing

Always run these commands before committing:

# 1. Check code formatting
npm run format:check

# 2. Run linting
npm run lint

# 3. Run E2E tests
npm run test:e2e

If any of these fail, fix the issues before pushing:

# Auto-fix formatting issues
npm run format

# Auto-fix linting issues
npm run lint:fix

For Contributors

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/amazing-feature
  3. Run tests locally (see above)
  4. Commit your changes: git commit -m 'Add amazing feature'
  5. Push to the branch: git push origin feature/amazing-feature
  6. Open a Pull Request

The CI pipeline will automatically:

  • Run linting checks
  • Verify code formatting
  • Execute E2E tests
  • Build Docker image
  • Report any failures

Testing

E2E Test Suite

The project includes comprehensive E2E tests:

# Run all E2E tests
npm run test:e2e

Tests include:

  • Health endpoint validation
  • S3 storage connectivity
  • Download initiation workflow
  • File availability checks
  • Error handling scenarios

Manual Testing

# 1. Check health status
curl http://localhost:3000/health

# Expected: {"status":"healthy","checks":{"storage":"ok"}}

# 2. Check file availability
curl -X POST http://localhost:3000/v1/download/check \
  -H "Content-Type: application/json" \
  -d '{"file_id": 70000}'

# 3. Test Sentry error tracking
curl -X POST "http://localhost:3000/v1/download/check?sentry_test=true" \
  -H "Content-Type: application/json" \
  -d '{"file_id": 70000}'

Challenge Solutions

Challenge 1: S3 Storage Integration

See implementation in docker/compose.dev.yml and docker/compose.prod.yml

Verification:

curl http://localhost:3000/health
# Must return: {"status":"healthy","checks":{"storage":"ok"}}

Challenge 2: Architecture Design

See ARCHITECTURE.md for complete system design, including:

  • Architecture diagrams
  • Pattern selection (Polling/WebSocket/Webhook)
  • Implementation details
  • Proxy configurations
  • Frontend integration guide

Challenge 3: CI/CD Pipeline

See .github/workflows/ci.yml for pipeline configuration.

Challenge 4: Observability Dashboard

See frontend/ directory for React dashboard implementation.

Troubleshooting

Common Issues

Issue: Port already in use

# Kill process using port 3000
npx kill-port 3000

# Or use a different port
PORT=3001 npm run dev

Issue: Docker services not starting

# Clean up and restart
npm run docker:down
docker system prune -f
npm run docker:dev

Issue: E2E tests failing

# Ensure server is not running
npx kill-port 3000

# Run tests with clean state
npm run test:e2e

License

This project is licensed under the MIT License.

Built with care for CUET Micro-Ops Hackathon 2025

About

No description, website, or topics provided.

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages