YouTube Link Processor

High-performance YouTube media extraction API with async processing

---

📋 Table of Contents

• Overview
• Features
• Quick Start
• Usage
• API Reference
• Environment Variables
• Tech Stack
• Architecture
• Contributing
• License
• Contact

---

📖 Overview

YouTube Link Processor is a production-grade REST API service for extracting and downloading media from YouTube URLs. Built with FastAPI, it leverages async/await patterns for high-throughput request handling, with background job processing via Redis queues for resource-intensive media extraction operations.

The system employs a decoupled architecture: the FastAPI application handles authentication, job management, and file delivery, while a dedicated worker process consumes jobs from Redis and performs media extraction using yt-dlp.

A modern HTMX-powered web interface provides users with a seamless experience for registering, logging in, and managing their downloads directly from the browser.

---

✨ Features

• 🔐 Secure Authentication — JWT-based auth with access/refresh token rotation
• ⚡ Async Processing — Non-blocking job queue with Redis-backed worker
• 📊 Job Lifecycle Management — Complete job states: pending → processing → completed/failed
• 🔗 Flexible Media Extraction — yt-dlp powered extraction for various formats
• ⏰ Expiring Downloads — Time-limited download links (configurable, default 24h)
• 🐳 Container-Ready — Multi-stage Docker builds with multi-arch support
• 🧪 Test Suite — Comprehensive pytest coverage with async test support
• 🌐 HTMX Frontend — Dynamic web UI with login, register, and downloads dashboard
• 📥 Download Dashboard — User-friendly interface to manage download jobs
• 🔄 Real-time Status — HTMX-powered polling for live job status updates

---

🚀 Quick Start

Prerequisites

• Python 3.12+
• PostgreSQL 14+
• Redis 7+

Option 1: Docker Compose (Recommended)

# Clone the repository
git clone https://github.com/yourusername/vooglaadija.git
cd vooglaadija

# Start all services
docker-compose up -d

Option 2: Local Development

# Clone and setup
git clone https://github.com/yourusername/vooglaadija.git
cd vooglaadija

# Install dependencies using hatch
hatch env create

# Or install with specific features
pip install "vooglaadija[test,lint]"

# Configure environment
cp .env.example .env
# Edit .env with your settings

# Run database migrations
hatch run db-migrate

# Run the API
hatch run dev

# Run the worker (separate terminal)
hatch run python -m worker.main

Access the Web Interface

Once running, access the HTMX frontend at:

• Home: http://localhost:8000/
• Login: http://localhost:8000/login
• Register: http://localhost:8000/register
• Downloads: http://localhost:8000/downloads (requires login)

---

💻 Usage

Register a new user

curl -X POST http://localhost:8000/api/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email": "user@example.com", "password": "securepassword123"}'

Login and obtain tokens

curl -X POST http://localhost:8000/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email": "user@example.com", "password": "securepassword123"}'

Response:

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "token_type": "bearer"
}

Create a download job

curl -X POST http://localhost:8000/api/v1/downloads \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"}'

Response:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "pending",
  "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
  "created_at": "2024-01-15T10:30:00Z"
}

Check job status

curl -X GET http://localhost:8000/api/v1/downloads/550e8400-e29b-41d4-a716-446655440000 \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Download the file

curl -X GET "http://localhost:8000/api/v1/downloads/550e8400-e29b-41d4-a716-446655440000/file" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -o downloaded_file.mp4

---

🔌 API Reference

Web UI Routes

Method	Endpoint	Description
GET	/	Home page
GET	/login	Login page
POST	/login	Login form submission
GET	/register	Registration page
POST	/register	Registration form submission
GET	/logout	Logout and redirect
GET	/downloads	Downloads dashboard
GET	/downloads/{id}/status	Job status (HTMX partial)

REST API Endpoints

Method	Endpoint	Description
POST	/api/v1/auth/register	Register new user account
POST	/api/v1/auth/login	Authenticate and get JWT tokens
POST	/api/v1/auth/refresh	Refresh access token
GET	/api/v1/me	Get current user profile
POST	/api/v1/downloads	Create new download job
GET	/api/v1/downloads	List user's download jobs
GET	/api/v1/downloads/{id}	Get job status and details
GET	/api/v1/downloads/{id}/file	Download the processed file
DELETE	/api/v1/downloads/{id}	Delete a download job
GET	/api/v1/health	Service health check

---

⚙️ Environment Variables

Variable	Description	Default
DATABASE_URL	PostgreSQL connection string	postgresql://user:pass@localhost:5432/ytprocessor
SECRET_KEY	JWT signing key	change-me
REDIS_URL	Redis connection string	redis://localhost:6379
CORS_ORIGINS	Allowed CORS origins	*
ACCESS_TOKEN_EXPIRE_MINUTES	Access token expiry (minutes)	15
REFRESH_TOKEN_EXPIRE_DAYS	Refresh token expiry (days)	7
FILE_EXPIRE_HOURS	Download link expiry (hours)	24
STORAGE_PATH	Local storage directory	./storage

---

🛠 Tech Stack

Technology	Purpose
	Runtime
	API Framework
	ORM
	Database
	Queue & Cache
	Containerization
	CI/CD

Key Libraries

• python-jose[cryptography] — JWT token handling
• passlib[bcrypt] — Password hashing
• yt-dlp — Media extraction engine
• pytest — Testing framework
• Jinja2 — Template engine for HTMX pages
• HTMX — Dynamic frontend interactions

Project Structure

app/
├── api/routes/
│   ├── pages.py         # HTMX page routes
│   └── downloads.py     # Download API endpoints
├── templates/           # Jinja2 templates
│   ├── base.html
│   ├── pages/          # Login, register, downloads
│   └── partials/       # Navbar, cards, forms
└── static/css/         # Stylesheets

---

🏗 Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Client    │────▶│  FastAPI    │────▶│ PostgreSQL  │
│ Application │     │     API     │     │  Database   │
└─────────────┘     └──────┬──────┘     └─────────────┘
                           │
                           ▼
                    ┌─────────────┐
                    │    Redis    │
                    │    Queue    │
                    └──────┬──────┘
                           │
                           ▼
                    ┌─────────────┐     ┌─────────────┐
                    │   Worker    │────▶│   Storage   │
                    │  (yt-dlp)   │     │  (downloads)│
                    └─────────────┘     └─────────────┘

Component Responsibilities

API Server

• User authentication (register, login, token refresh)
• Download job CRUD operations
• File streaming and expiration logic
• Health check endpoints

Worker Process

• Consumes jobs from Redis queue
• Extracts media using yt-dlp
• Updates job status in PostgreSQL
• Manages file lifecycle

---

🤝 Contributing

We welcome contributions! Please follow these guidelines:

Development Workflow

1. Fork the repository
2. Create a feature branch: feat/your-feature or fix/your-bug
3. Commit your changes following conventional commits
4. Push to your fork and create a Pull Request
5. Ensure all tests pass before merging

Week-by-Week Workflow

This is designed as an 8-week working group project. Each week has specific deliverables:

• Week 1-2: Complete foundation and authentication UI
• Week 3-4: Complete downloads dashboard and status polling
• Week 5-7: Polish and testing
• Week 8: Cloud deployment (if required)

Branch Strategy

• main — Production-ready code only
• Feature branches — Short-lived, deleted after merge
• Direct commits to main are prohibited

Code Standards

• Type hints required for all new code
• 100% test coverage for new features
• Follow PEP 8 with 100 character line limit
• Use async/await for I/O-bound operations

Running Tests

# Run all tests with coverage
pytest tests/ --cov=app --cov-report=term-missing

# Run specific test file
pytest tests/test_api/test_auth.py -v

Manual Test Checklist

Before demoing, verify:

• Register new user works
• Login with wrong password shows error
• Login with correct password redirects
• Create download with invalid URL shows error
• Create download with valid URL appears in list
• Status updates during processing
• Completed file downloads successfully
• Logout redirects to login

---

📜 License

This project is licensed under the GNU General Public License v3.0.

YouTube Link Processor - REST API for YouTube media extraction
Copyright (C) 2024

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

---

📞 Contact

Built with FastAPI • Powered by yt-dlp