Skip to content

INSTALLATION.md

Latest commit

 

History

History
465 lines (331 loc) · 9.73 KB

INSTALLATION.md

File metadata and controls

465 lines (331 loc) · 9.73 KB

Installation Guide

Complete guide for setting up OpenProcessor on your system.

TL;DR - One Line Setup

git clone https://github.com/davidamacey/OpenProcessor.git && cd OpenProcessor && ./scripts/setup.sh

Quick Start (Recommended)

Interactive Setup

git clone https://github.com/davidamacey/OpenProcessor.git
cd OpenProcessor
./scripts/setup.sh

Non-Interactive Setup

# Accept all defaults
./scripts/setup.sh --yes

# Specify profile and GPU
./scripts/setup.sh --profile=standard --gpu=0 --yes

# Skip TensorRT export (if models already exported)
./scripts/setup.sh --skip-export --yes

What Setup Does

The setup script will:

  1. Check prerequisites (Docker, NVIDIA drivers)
  2. Detect your GPU and select the optimal profile
  3. Pull pre-built Docker images from Docker Hub (~32GB)
  4. Download required models (~500MB, ~16 seconds)
  5. Export models to TensorRT (~30-60 minutes, one-time only)
  6. Generate configuration files
  7. Start all services
  8. Run smoke tests

First-time setup takes ~30-60 minutes (mostly TensorRT compilation). Subsequent starts take ~30 seconds since compiled engines are cached on disk.

Prerequisites

Required Software

Software Minimum Version Installation
Docker 20.10+ docs.docker.com
Docker Compose v2.0+ Included with Docker Desktop
NVIDIA Driver 535+ nvidia.com/drivers
NVIDIA Container Toolkit Latest See below

Hardware Requirements

Requirement Minimum Recommended
GPU VRAM 6GB 12GB+
GPU Architecture Ampere (30-series) Ampere or newer
System RAM 16GB 32GB+
CPU Cores 8 16+
Storage 20GB free 50GB+ (SSD recommended)

NVIDIA Container Toolkit

Install the NVIDIA Container Toolkit to enable GPU access in Docker:

# Ubuntu/Debian
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configure Docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Verify toolkit is configured
docker info | grep -i nvidia
nvidia-smi

GPU Profiles

The system automatically selects a profile based on your GPU's VRAM:

Profile VRAM Range Models Performance Use Case
minimal 6-8GB Core only ~5 RPS RTX 3060, RTX 4060
standard 12-24GB All ~15 RPS RTX 3080, RTX 4090
full 48GB+ All ~50 RPS A6000, A100

Profile Details

minimal (6-8GB GPUs):

  • YOLO object detection
  • SCRFD face detection
  • ArcFace embeddings
  • MobileCLIP embeddings
  • No OCR (optional add-on)
  • 1 instance per model, batch size 16

standard (12-24GB GPUs):

  • All models including OCR
  • 2 instances per model, batch size 32
  • Good for most production workloads

full (48GB+ GPUs):

  • All models with maximum parallelism
  • 4 instances per model, batch size 64
  • High-throughput production systems

Manually Selecting a Profile

# During setup
./scripts/setup.sh --profile=minimal

# After setup
./scripts/openprocessor.sh profile minimal
./scripts/openprocessor.sh restart

Manual Installation

If you prefer to run steps individually:

1. Clone Repository

git clone https://github.com/davidamacey/OpenProcessor.git
cd OpenProcessor

2. Create Directories

mkdir -p pytorch_models logs cache/huggingface outputs test_results

3. Download Models

Models are downloaded from public sources (no authentication required):

# All models (~500MB)
./scripts/openprocessor.sh download all

# Or essential only (~250MB)
./scripts/openprocessor.sh download essential

# Check download status
./scripts/openprocessor.sh download status

4. Configure Environment

Copy and edit the environment template:

cp env.template .env
# Edit .env to adjust settings

Or generate automatically:

./scripts/openprocessor.sh profile standard

5. Start Containers (for export)

docker compose up -d triton-server yolo-api

6. Export to TensorRT

This step converts models to optimized TensorRT format:

# All models (45-60 minutes)
./scripts/openprocessor.sh export all

# Or essential only (25-35 minutes)
./scripts/openprocessor.sh export essential

# Check export status
./scripts/openprocessor.sh export status

7. Start All Services

docker compose up -d

8. Verify Installation

# Check status
./scripts/openprocessor.sh status

# Run smoke tests
./scripts/openprocessor.sh test quick

# Test API directly
curl http://localhost:4603/health

Troubleshooting

GPU Out of Memory (OOM)

Symptoms: Services crash, "CUDA out of memory" errors

Solutions:

  1. Switch to a smaller profile:

    ./scripts/openprocessor.sh profile minimal
./scripts/openprocessor.sh restart

  2. Reduce batch size in .env:

    MAX_BATCH_SIZE=8

  3. Stop other GPU processes:

    nvidia-smi  # Check what's using GPU

TensorRT Export Fails

Symptoms: Export hangs or errors during trtexec

Solutions:

  1. Check GPU memory is available:

    nvidia-smi  # Need 4GB+ free

  2. Unload existing models first:

    docker compose stop triton-server
./scripts/openprocessor.sh export all
docker compose start triton-server

  3. Check CUDA version compatibility:

    nvidia-smi  # Driver version
docker compose exec triton-server nvidia-smi  # Container CUDA version

Models Not Loading

Symptoms: Triton shows "UNAVAILABLE" for models

Solutions:

  1. Check if exports exist:

    ./scripts/openprocessor.sh export status

  2. Check Triton logs:

    ./scripts/openprocessor.sh logs triton-server

  3. Verify model configs:

    ls -la models/*/config.pbtxt

Docker Permission Errors

Symptoms: "Permission denied" when running Docker commands

Solutions:

  1. Add user to docker group:

    sudo usermod -aG docker $USER
# Log out and back in

  2. Check Docker socket permissions:

    ls -la /var/run/docker.sock

Services Won't Start

Symptoms: docker compose up fails

Solutions:

  1. Check port conflicts:

    lsof -i :4603  # API port
lsof -i :4600  # Triton port

  2. Check disk space:

    df -h

  3. View detailed errors:

    docker compose up  # Without -d to see output

Upgrading

Pulling Latest Version

git pull
./scripts/openprocessor.sh update
./scripts/openprocessor.sh restart

Rebuilding After Updates

docker compose build --no-cache
docker compose up -d

Re-exporting Models

After major updates, you may need to re-export TensorRT models:

./scripts/openprocessor.sh export all
./scripts/openprocessor.sh restart

Uninstallation

Stop and Remove Containers

docker compose down -v  # -v removes volumes

Remove Images

docker compose down --rmi all

Clean Up Files

# Remove generated files
rm -rf pytorch_models/*.pt pytorch_models/*.onnx
rm -rf models/*/1/model.plan
rm -rf .env docker-compose.override.yml
rm -rf cache/ logs/ outputs/ test_results/

Docker Image Building

Production Dockerfiles

The project includes production-optimized Dockerfiles:

File Purpose Base Image
Dockerfile FastAPI service python:3.13-slim-trixie
Dockerfile.triton Triton server nvcr.io/nvidia/tritonserver:25.10-py3

Building Images

# Build and push to Docker Hub
./scripts/docker-build-push.sh all

# Build locally without pushing
./scripts/docker-build-push.sh local

# Build with specific version
VERSION=v1.0.0 ./scripts/docker-build-push.sh all

Security Scanning

The project includes comprehensive security scanning using free, open-source tools:

Tools Used:

  • Hadolint - Dockerfile linting
  • Dockle - CIS Docker Benchmark compliance
  • Trivy - Vulnerability scanning
  • Grype - Fast vulnerability scanning
  • Syft - SBOM (Software Bill of Materials) generation
# Install security scanning tools
./scripts/security-scan.sh install

# Scan all images
./scripts/security-scan.sh all

# Scan specific image
./scripts/security-scan.sh api

# View reports
ls -la security-reports/

Reports Generated:

  • *-hadolint.txt - Dockerfile linting results
  • *-dockle.json - CIS best practices check
  • *-sbom.json - Software Bill of Materials (CycloneDX)
  • *-trivy.json/txt - Vulnerability scan results
  • *-grype.json/txt - Additional vulnerability scan

CI/CD Integration

For automated builds in CI/CD pipelines:

# Fail on security issues
FAIL_ON_SECURITY_ISSUES=true FAIL_ON_CRITICAL=true ./scripts/docker-build-push.sh all

# Skip scanning for faster builds
SKIP_SECURITY_SCAN=true ./scripts/docker-build-push.sh all

Getting Help