Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ The format is based on [Common Changelog](https://common-changelog.org/).

### Added

- **MySQL Documentation** ([#246](https://github.com/sofatutor/llm-proxy/pull/246)): Comprehensive documentation updates for MySQL support including README updates, database selection guide, Helm deployment docs, configuration reference, architecture docs, and production HA guidance so users can confidently deploy and operate llm-proxy with MySQL alongside SQLite and PostgreSQL.
- **MySQL Helm Chart Support** ([#255](https://github.com/sofatutor/llm-proxy/pull/255)): Adds comprehensive MySQL support to the Helm chart with values, templates, helper functions, deployment integration, and updated docs so users can deploy llm-proxy with MySQL alongside SQLite and PostgreSQL.
- **MySQL Compose Profile** ([#253](https://github.com/sofatutor/llm-proxy/pull/253)): Added optional MySQL backend with dedicated Docker Compose services, build flags, docs, and environment vars so teams can run isolated dev and test stacks alongside existing databases.
- **MySQL Integration Suite** ([#256](https://github.com/sofatutor/llm-proxy/pull/256)): Introduced comprehensive MySQL integration tests and helpers covering connections, migrations, CRUD operations, concurrency, transactions, and stats so the suite can be exercised locally and validated in CI.
Expand Down
17 changes: 13 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ A transparent, secure proxy for OpenAI's API with token management, rate limitin
- **Async Event Bus & Dispatcher**: All API instrumentation events are handled via an always-on, fully asynchronous event bus (in-memory or Redis) with support for multiple subscribers, batching, retry logic, and graceful shutdown. Persistent event logging is handled by a dispatcher CLI or the `--file-event-log` flag.
- **OpenAI Token Counting**: Accurate prompt and completion token counting using tiktoken-go.
- **Metrics Endpoint (provider-agnostic)**: Optional JSON metrics endpoint; Prometheus scraping/export is optional and not required by core features
- **Multiple Database Support**: SQLite (default) and PostgreSQL with automatic migrations
- **Multiple Database Support**: SQLite (default), PostgreSQL, and MySQL with automatic migrations
- **Database Migrations**: Version-controlled schema changes with rollback support. See [Migration Guide](docs/database/migrations.md)
- **Docker Deployment**

Expand Down Expand Up @@ -101,7 +101,7 @@ MANAGEMENT_TOKEN=your-secure-management-token ./bin/llm-proxy
- `FILE_EVENT_LOG`: Path to persistent event log file (enables file event logging via dispatcher)

### Database Configuration
The LLM Proxy supports **SQLite** (default) and **PostgreSQL** as database backends.
The LLM Proxy supports **SQLite** (default), **PostgreSQL**, and **MySQL** as database backends.

**SQLite (default):**
- `DB_DRIVER`: Database driver, set to `sqlite` (default)
Expand All @@ -111,12 +111,21 @@ The LLM Proxy supports **SQLite** (default) and **PostgreSQL** as database backe
- `DB_DRIVER`: Set to `postgres` for PostgreSQL
- `DATABASE_URL`: PostgreSQL connection string (e.g., `postgres://user:password@localhost:5432/llmproxy?sslmode=require`)

**Connection Pool (both drivers):**
**MySQL:**
- `DB_DRIVER`: Set to `mysql` for MySQL
- `DATABASE_URL`: MySQL connection string (e.g., `llmproxy:secret@tcp(localhost:3306)/llmproxy?parseTime=true&tls=true`)

**Connection Pool (all drivers):**
- `DATABASE_POOL_SIZE`: Maximum open connections (default `10`)
- `DATABASE_MAX_IDLE_CONNS`: Maximum idle connections (default `5`)
- `DATABASE_CONN_MAX_LIFETIME`: Connection max lifetime (default `1h`)

See [PostgreSQL Setup Guide](docs/database/docker-compose-postgres.md) for detailed PostgreSQL configuration.
**Build Requirements:**
- SQLite: No build tags required (included by default)
- PostgreSQL: Requires `-tags postgres` build flag
- MySQL: Requires `-tags mysql` build flag

See [Database Selection Guide](docs/database/database-selection.md) for choosing the right database, or the [PostgreSQL Setup Guide](docs/database/docker-compose-postgres.md) and [MySQL Setup Guide](docs/database/docker-compose-mysql.md) for detailed configuration.

### Caching Configuration
- `HTTP_CACHE_ENABLED`: Enable HTTP response caching (default `true`)
Expand Down
16 changes: 14 additions & 2 deletions deploy/helm/llm-proxy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,13 @@ helm install llm-proxy deploy/helm/llm-proxy \
```

**IMPORTANT:**
- In-cluster PostgreSQL is for development/testing only, NOT recommended for production
- **⚠️ For production high-availability (HA) deployments, use an external managed PostgreSQL service:**
- Amazon Aurora PostgreSQL / Amazon RDS for PostgreSQL
- Google Cloud SQL for PostgreSQL
- Azure Database for PostgreSQL
- Self-managed PostgreSQL with replication
- The in-cluster PostgreSQL StatefulSet is hardcoded to `replicas: 1` and is suitable for **development/testing only**
- It does not provide high availability or automatic failover
- Ensure your Docker image is built with PostgreSQL support using the `postgres` build tag
- Default images are built with: `docker build --build-arg POSTGRES_SUPPORT=true`
- You cannot use both in-cluster and external PostgreSQL simultaneously
Expand Down Expand Up @@ -261,7 +267,13 @@ helm install llm-proxy deploy/helm/llm-proxy \
```

**IMPORTANT:**
- In-cluster MySQL is for development/testing only, NOT recommended for production
- **⚠️ For production high-availability (HA) deployments, use an external managed MySQL service:**
- Amazon Aurora MySQL / Amazon RDS for MySQL
- Google Cloud SQL for MySQL
- Azure Database for MySQL
- Self-managed MySQL Group Replication or InnoDB Cluster
- The in-cluster MySQL StatefulSet is hardcoded to `replicas: 1` and is suitable for **development/testing only**
- It does not provide high availability or automatic failover
- Ensure your Docker image is built with MySQL support using the `mysql` build tag
- Build command: `docker build --build-arg MYSQL_SUPPORT=true -t llm-proxy:mysql .`
- You cannot use both in-cluster and external MySQL simultaneously
Expand Down
7 changes: 4 additions & 3 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ Start with the main [README](../README.md) for a quick overview, then follow the
- **[API Configuration](guides/api-configuration.md)** - Configure API providers, endpoints, and security policies
- **[Security Best Practices](deployment/security.md)** - Production security, secrets management, and hardening
- **[Docker Compose PostgreSQL Setup](database/docker-compose-postgres.md)** - Run llm-proxy with PostgreSQL using Docker Compose
- **[Database Selection Guide](database/database-selection.md)** - Choose between SQLite and PostgreSQL
- **[Docker Compose MySQL Setup](database/docker-compose-mysql.md)** - Run llm-proxy with MySQL using Docker Compose
- **[Database Selection Guide](database/database-selection.md)** - Choose between SQLite, PostgreSQL, and MySQL
- **[Database Migrations Guide](database/migrations.md)** - Version-controlled schema changes

## Observability & Monitoring
Expand Down Expand Up @@ -87,9 +88,9 @@ The [OpenAPI specification](../api/openapi.yaml) provides machine-readable API d
|----------|---------|---------|
| `MANAGEMENT_TOKEN` | Admin API access | **Required** |
| `LISTEN_ADDR` | Server address | `:8080` |
| `DB_DRIVER` | Database driver (`sqlite` or `postgres`) | `sqlite` |
| `DB_DRIVER` | Database driver (`sqlite`, `postgres`, or `mysql`) | `sqlite` |
| `DATABASE_PATH` | SQLite database path | `./data/llm-proxy.db` |
| `DATABASE_URL` | PostgreSQL connection string | - |
| `DATABASE_URL` | PostgreSQL or MySQL connection string | - |
| `LOG_LEVEL` | Logging level | `info` |
| `HTTP_CACHE_ENABLED` | Enable response caching | `true` |
| `HTTP_CACHE_BACKEND` | Cache backend (`in-memory` or `redis`) | `in-memory` |
Expand Down
17 changes: 10 additions & 7 deletions docs/architecture/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,9 +365,11 @@ The proxy implementation is based on Go's `httputil.ReverseProxy` with customiza
- Tokens table: Stores tokens with expiration and usage limits
- **Implementation**: `internal/database/*`
- **Technology**:
- **SQLite** is used for MVP, local development, and small-scale/self-hosted deployments for its simplicity and zero-dependency deployment.
- **PostgreSQL** is recommended for production deployments requiring high concurrency, advanced features, or distributed/cloud-native scaling.
- The codebase and schema/migrations are designed to support both SQLite and PostgreSQL, enabling a smooth migration path as needed.
- **SQLite** (default): Zero-dependency option for local development, MVP, and small-scale/self-hosted deployments
- **PostgreSQL**: Recommended for production deployments requiring high concurrency, advanced features, or distributed/cloud-native scaling
- **MySQL**: Recommended for production deployments, especially for teams with existing MySQL infrastructure or expertise
- The codebase and schema/migrations are designed to support all three databases, enabling a smooth migration path as needed
- Build tags (`-tags postgres` or `-tags mysql`) control which database drivers are compiled into the binary

### Token Management

Expand Down Expand Up @@ -570,7 +572,8 @@ sequenceDiagram
The application is designed for flexible deployment:

- For MVP, local, and small-scale deployments, a single container with SQLite is recommended for simplicity.
- For production or scaling needs, PostgreSQL can be used as the backing database, either in a container or as a managed service. The application should be configured to connect to PostgreSQL as needed.
- For production or scaling needs, PostgreSQL or MySQL can be used as the backing database, either in a container or as a managed service. The application should be configured to connect to the chosen database as needed.
- Build tags must be used when compiling with PostgreSQL (`-tags postgres`) or MySQL (`-tags mysql`) support.

### Single Container Deployment

Expand Down Expand Up @@ -615,9 +618,9 @@ flowchart TD
Proxy2 --> Redis
Proxy3 --> Redis

Proxy1 --> Postgres[("PostgreSQL")]
Proxy2 --> Postgres
Proxy3 --> Postgres
Proxy1 --> Database[("PostgreSQL/MySQL")]
Proxy2 --> Database
Proxy3 --> Database

Proxy1 --> API["Target API"]
Proxy2 --> API
Expand Down
132 changes: 119 additions & 13 deletions docs/database/database-selection.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,18 @@ This guide helps you choose between SQLite and PostgreSQL for your LLM Proxy dep

## Overview

The LLM Proxy supports two database backends:

| Feature | SQLite | PostgreSQL |
|---------|--------|------------|
| Setup Complexity | Zero configuration | Requires server setup |
| Scalability | Single instance | Multiple instances |
| Concurrent Writes | Limited | Excellent |
| Best For | Development, small deployments | Production, high-traffic |
| Maintenance | None | Requires DBA knowledge |
The LLM Proxy supports three database backends:

| Feature | SQLite | PostgreSQL | MySQL |
|---------|--------|------------|-------|
| Setup Complexity | Zero configuration | Requires server setup | Requires server setup |
| Scalability | Single instance | Multiple instances | Multiple instances |
| Concurrent Writes | Limited | Excellent | Excellent |
| Best For | Development, small deployments | Production, high-traffic | Production, web apps |
| Build Tag Required | No | Yes (`-tags postgres`) | Yes (`-tags mysql`) |
| Maintenance | None | Requires DBA knowledge | Requires DBA knowledge |
| Cloud Managed Options | N/A | RDS, Aurora, Cloud SQL, Azure | RDS, Aurora, Cloud SQL, Azure |
| In-Cluster Helm | N/A | Dev/Test only (single replica) | Dev/Test only (single replica) |

## SQLite (Default)

Expand Down Expand Up @@ -112,9 +115,100 @@ DATABASE_URL=postgres://user@server:pass@server.postgres.database.azure.com:5432
- **Concurrent connections**: Thousands with proper configuration
- **Recommended**: 100+ requests/second, multiple instances

### Production Deployment

**⚠️ For production high-availability (HA) deployments, use an external managed PostgreSQL service:**
- Amazon Aurora PostgreSQL / Amazon RDS for PostgreSQL
- Google Cloud SQL for PostgreSQL
- Azure Database for PostgreSQL
- Self-managed PostgreSQL with replication

**Important:** The in-cluster PostgreSQL StatefulSet in the Helm chart is hardcoded to `replicas: 1` and is suitable for **development/testing only**. It does not provide high availability or automatic failover.

## MySQL

MySQL is recommended for production deployments, especially for teams already familiar with MySQL or using MySQL-based infrastructure.

### When to Use MySQL

- **Production deployments** - Excellent reliability and wide ecosystem
- **High traffic** - Handles thousands of concurrent connections
- **Multiple instances** - Required for horizontal scaling
- **MySQL familiarity** - Team has existing MySQL expertise
- **MySQL infrastructure** - Already using MySQL for other services

### Configuration

```bash
export DB_DRIVER=mysql
export DATABASE_URL=llmproxy:secret@tcp(localhost:3306)/llmproxy?parseTime=true&tls=true

# Optional: Connection pool settings
export DATABASE_POOL_SIZE=10
export DATABASE_MAX_IDLE_CONNS=5
export DATABASE_CONN_MAX_LIFETIME=1h
```

### Connection String Format

```
[user]:[password]@tcp([host]:[port])/[database]?parseTime=true&tls=[mode]
```

**TLS/SSL Options:**

| Parameter | Description | Use Case |
|-----------|-------------|----------|
| `tls=false` | No TLS | Development only |
| `tls=true` | TLS with system CA verification | Production (recommended) |
| `tls=skip-verify` | TLS without certificate verification | Not recommended |
| `tls=custom` | TLS with custom CA | Advanced configurations |

**Examples:**

```bash
# Local development (no TLS)
DATABASE_URL=llmproxy:secret@tcp(localhost:3306)/llmproxy?parseTime=true

# Production with TLS
DATABASE_URL=llmproxy:secret@tcp(localhost:3306)/llmproxy?parseTime=true&tls=true

# AWS RDS MySQL
DATABASE_URL=admin:password@tcp(mydb.xxx.rds.amazonaws.com:3306)/llmproxy?parseTime=true&tls=true

# Google Cloud SQL MySQL
DATABASE_URL=root:password@tcp(10.x.x.x:3306)/llmproxy?parseTime=true&tls=true

# Azure Database for MySQL
DATABASE_URL=admin@server:password@tcp(server.mysql.database.azure.com:3306)/llmproxy?parseTime=true&tls=true
```

### Performance Characteristics

- **Read performance**: Excellent with proper indexing
- **Write performance**: Excellent with connection pooling
- **Concurrent connections**: Thousands with proper configuration
- **Recommended**: 100+ requests/second, multiple instances

### Compatibility

- **Minimum Version**: MySQL 8.0+
- **MariaDB Support**: MariaDB 10.5+ is compatible
- **Build Requirement**: Binary must be built with `-tags mysql` flag

### Production Deployment

**⚠️ For production high-availability (HA) deployments, use an external managed MySQL service:**
- Amazon Aurora MySQL / Amazon RDS for MySQL
- Google Cloud SQL for MySQL
- Azure Database for MySQL
- Self-managed MySQL Group Replication or InnoDB Cluster

**Important:** The in-cluster MySQL StatefulSet in the Helm chart is hardcoded to `replicas: 1` and is suitable for **development/testing only**. It does not provide high availability or automatic failover.

## Migration Between Databases

### SQLite to PostgreSQL
### SQLite to PostgreSQL or MySQL

1. **Export data from SQLite:**

Expand All @@ -123,17 +217,26 @@ DATABASE_URL=postgres://user@server:pass@server.postgres.database.azure.com:5432
sqlite3 data/llm-proxy.db ".dump tokens" > tokens.sql
```

2. **Start PostgreSQL:**
2. **Start target database:**

```bash
# PostgreSQL
docker compose --profile postgres up -d postgres

# MySQL
docker compose --profile mysql up -d mysql
```

3. **Update configuration:**

```bash
# For PostgreSQL
export DB_DRIVER=postgres
export DATABASE_URL=postgres://llmproxy:secret@localhost:5432/llmproxy?sslmode=disable

# For MySQL
export DB_DRIVER=mysql
export DATABASE_URL=llmproxy:secret@tcp(localhost:3306)/llmproxy?parseTime=true
```

4. **Start the proxy** (migrations run automatically):
Expand All @@ -145,15 +248,16 @@ DATABASE_URL=postgres://user@server:pass@server.postgres.database.azure.com:5432
5. **Import data** (requires SQL syntax adaptation):

```bash
# Note: SQLite and PostgreSQL SQL dialects differ
# Note: SQLite, PostgreSQL, and MySQL SQL dialects differ
# Manual adjustment of exported SQL may be required
```

### Considerations

- **Schema migrations**: Run automatically on startup for both databases
- **Schema migrations**: Run automatically on startup for all databases
- **Data migration**: Manual process, requires SQL dialect conversion
- **Downtime**: Plan for maintenance window during migration
- **Build tags**: Ensure binary is built with appropriate tags (`-tags postgres` or `-tags mysql`)

## Connection Pooling

Expand Down Expand Up @@ -219,5 +323,7 @@ See the [PostgreSQL Troubleshooting Guide](postgresql-troubleshooting.md) for co
## Related Documentation

- [Docker Compose PostgreSQL Setup](docker-compose-postgres.md)
- [Docker Compose MySQL Setup](docker-compose-mysql.md)
- [Database Migrations Guide](migrations.md)
- [Migration Concurrency](migrations-concurrency.md)
- [PostgreSQL Troubleshooting Guide](postgresql-troubleshooting.md)
9 changes: 9 additions & 0 deletions docs/database/docker-compose-mysql.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,15 @@ MySQL named locking requires the 'mysql' build tag

The llm-proxy supports SQLite (default), PostgreSQL, and MySQL as database backends. This guide explains how to run the MySQL setup using Docker Compose.

**⚠️ Important for Production:**
- The Docker Compose MySQL setup is intended for **development and testing only**
- For production high-availability (HA) deployments, use an external managed MySQL service:
- Amazon Aurora MySQL / Amazon RDS for MySQL
- Google Cloud SQL for MySQL
- Azure Database for MySQL
- Self-managed MySQL Group Replication or InnoDB Cluster
- The Docker Compose MySQL instance runs as a single container and does not provide high availability or automatic failover

## Quick Start

### Prerequisites
Expand Down
11 changes: 10 additions & 1 deletion docs/database/docker-compose-postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,16 @@ PostgreSQL support not compiled in; build with -tags postgres to enable

## Overview

The llm-proxy supports both SQLite (default) and PostgreSQL as database backends. This guide explains how to run the PostgreSQL setup using Docker Compose.
The llm-proxy supports SQLite (default), PostgreSQL, and MySQL as database backends. This guide explains how to run the PostgreSQL setup using Docker Compose.

**⚠️ Important for Production:**
- The Docker Compose PostgreSQL setup is intended for **development and testing only**
- For production high-availability (HA) deployments, use an external managed PostgreSQL service:
- Amazon Aurora PostgreSQL / Amazon RDS for PostgreSQL
- Google Cloud SQL for PostgreSQL
- Azure Database for PostgreSQL
- Self-managed PostgreSQL with replication
- The Docker Compose PostgreSQL instance runs as a single container and does not provide high availability or automatic failover

## Quick Start

Expand Down
Loading