Poser: Advanced Skier Pose Analysis Pipeline

Poser is a full-stack video analysis system for skier pose detection, turn segmentation, and technique metrics. It ships as a web app (React + FastAPI), a GPU-enabled analysis service, and a standalone CLI for local runs.

Visual Example

Landing Page	Results Dashboard

Key Features

• End-to-end web workflow: Upload, analyze, and review results from a browser with live status updates.
• Embeddable partner widget: Hosted JS widget with domain allowlisting and email confirmation flow.
• Pose analysis pipeline: YOLOv11 + BoT-SORT tracking, MediaPipe 3D pose, and temporal smoothing.
• Technique metrics: Edge similarity scoring and turn segmentation surfaced in the UI.
• Artifact exports: Pose overlay video, tracking overlay, CSV metrics, and optional 3D debug files.
• GPU acceleration: CUDA on Linux and MPS on Apple Silicon, with a native-analysis workflow for Macs.

Documentation

• Pipeline details and metrics: docs/analysis-pipeline.md
• Command line usage: docs/command-line.md
• Embed widget flow: docs/embed-widget.md

Processing Pipeline (High Level)

1. Detect and track skiers in video frames.
2. Estimate 3D pose landmarks for the primary track.
3. Smooth landmarks in time and compute metrics.
4. Render outputs and publish artifacts.

See docs/analysis-pipeline.md for the full breakdown.

Repository Structure

poser/
├── analysis/                 # Python analysis pipeline + service
│   ├── src/                  # Detection, pose, metrics, filters, visualization
│   ├── configs/              # YAML configs (runtime tuning)
│   ├── models/               # Downloaded model weights
│   ├── tests/                # Analysis tests
│   ├── analyze.py            # Headless analyzer (CLI + service entry)
│   ├── track.py              # Interactive CLI
│   └── server.py             # FastAPI analysis service
├── backend/                  # FastAPI API server (auth, storage, jobs)
│   ├── app/                  # Routes, config, DB models
│   ├── alembic/              # DB migrations
│   └── tests/                # Backend tests
├── frontend/                 # React + Vite frontend + embed widget
│   ├── src/                  # UI, pages, embed widget
│   └── dist/                 # Built assets (generated)
├── docs/                     # Product + engineering docs
├── docker-compose.yml        # Local dev services
├── fly.web.toml              # Fly.io config for poser-web
├── fly.analysis.toml         # Fly.io config for poser-analysis
├── Caddyfile.local           # Local reverse proxy rules
├── tests/                    # Repo-level tests (e.g., Caddyfile)
├── input/                    # Local sample inputs
├── output/                   # Local analysis outputs
└── scratch/                  # Local experiments

Development

Docker Compose (local stack)

Run the web app, database, and proxy locally:

docker compose up

Start the analysis container as well:

COMPOSE_PROFILES=analysis docker compose up

Note: local uploads require S3 credentials in .env (Tigris or a compatible bucket).

Local URLs:

• App: http://localhost
• Backend: http://localhost:8000
• API docs: http://localhost:8000/docs

Native analysis on Apple Silicon (GPU acceleration)

CUDA is not supported in Docker on Macs, so run the analysis service on the host for MPS:

export ANALYSIS_SERVICE_URL=http://host.docker.internal:8080
docker compose up -d db web caddy

source analysis/setup_native_env.sh
cd analysis
python server.py

Configuration

The analysis defaults live in analysis/configs/default.yaml and include inline comments. Edit that file (or pass --config) to tweak device selection, model paths, and smoothing values.

Container Architecture

The same topology is used locally and on Fly; analysis runs in a container or on the host, depending on your setup.

Browser
  │
  ▼
Edge Proxy
  - Local: Caddy (`Caddyfile.local`)
  - Production: Fly edge router
  │
  ▼
poser-web (FastAPI + static frontend)
  │  ├─ reads/writes: poser-db (Postgres)
  │  ├─ presigned uploads/downloads: Tigris S3
  │  └─ triggers: poser-analysis (/analyze)
  │
  ▼
poser-analysis (GPU service)
  ├─ downloads input from Tigris S3
  ├─ runs analysis/analyze.py
  └─ posts progress + artifacts to /api/internal/* on poser-web

Public endpoints (poser-web)

• POST /api/auth/request-code
• POST /api/auth/verify-code
• POST /api/analyses/create-upload
• POST /api/analyses/{id}/confirm-upload
• GET /api/analyses
• GET /api/analyses/{id}
• GET /api/analyses/{id}/edge-similarity
• GET /api/analyses/{id}/turns
• GET /api/analyses/{id}/download/{artifact_kind}
• DELETE /api/analyses/{id}
• POST /api/contact
• GET /api/embed/{partner_slug}/config
• POST /api/embed/{partner_slug}/submit
• POST /api/embed/{partner_slug}/upload-complete
• GET /api/embed/{partner_slug}/status/{analysis_id}
• GET /api/embed/{partner_slug}/feedback/{analysis_id}
• GET /api/embed/confirm?token=...
• GET /api/embed/results/{token}

Internal endpoints (analysis → backend)

• PUT /api/internal/analyses/{id}/progress
• PUT /api/internal/analyses/{id}/status
• PUT /api/internal/analyses/{id}/edge-similarity
• POST /api/internal/analyses/{id}/turns
• POST /api/internal/artifacts

Database Schema

┌──────────────────────────────┐         ┌──────────────────────────────┐
│            users             │         │      verification_codes      │
├──────────────────────────────┤         ├──────────────────────────────┤
│ id (PK)                      │         │ id (PK)                      │
│ email (unique)               │◀────────│ email                        │
│ verified_at                  │         │ code (bcrypt hash)           │
│ created_at                   │         │ expires_at                   │
│ updated_at                   │         │ used                         │
└──────────────────────────────┘         │ created_at                   │
              │                          └──────────────────────────────┘
              │ 1:N
              ▼
┌──────────────────────────────┐         ┌──────────────────────────────┐
│           analyses           │         │            uploads           │
├──────────────────────────────┤         ├──────────────────────────────┤
│ id (PK)                      │         │ id (PK)                      │
│ user_id (FK)                 │◀────────│ user_id (FK)                 │
│ partner_id (FK)              │         │ analysis_id (FK)             │
│ status                       │────────▶│ key (unique)                 │
│ filename                     │         │ size, mime, sha256           │
│ progress (JSON)              │         │ status                       │
│ analysis_results (JSON)      │         │ created_at, updated_at       │
│ edge_similarity (JSONB)      │         └──────────────────────────────┘
│ edge_similarity_* (float)    │
│ turns (JSONB)                │         ┌──────────────────────────────┐
│ s3_input_key                 │         │           artifacts          │
│ trim_start_seconds           │         ├──────────────────────────────┤
│ trim_end_seconds             │         │ id (PK)                      │
│ confirmed_at                 │         │ analysis_id (FK)             │
│ error_log                    │         │ key (unique)                 │
│ created_at, updated_at       │         │ kind, size, mime, sha256      │
└──────────────────────────────┘         │ created_at                   │
              │ 1:N                    ▲ └──────────────────────────────┘
              ▼                        │
┌──────────────────────────────┐        │
│           partners           │        │
├──────────────────────────────┤        │
│ id (PK)                      │        │
│ user_id (FK)                 │────────┘
│ slug (unique)                │
│ domain                       │
│ created_at, updated_at       │
└──────────────────────────────┘

┌──────────────────────────────┐
│    embed_confirmation_tokens │
├──────────────────────────────┤
│ id (PK)                      │
│ user_id (FK)                 │
│ analysis_id (FK)             │
│ token_hash (unique)          │
│ expires_at                   │
│ used                         │
│ created_at                   │
└──────────────────────────────┘

Production & CI/CD

Fly.io deployment

• poser-web: Combined backend + frontend (static assets built into the image).
• poser-analysis: GPU analysis service (auto-start/auto-stop).
• poser-db: Fly Postgres cluster attached to poser-web.
• Object storage: Tigris S3-compatible bucket for uploads and artifacts.

Communication in production:

• Browser ↔ poser-web for auth, uploads, results, and the embed widget (static assets are served by poser-web).
• poser-web ↔ poser-analysis via Flycast (ANALYSIS_SERVICE_URL).
• poser-analysis ↔ poser-web internal endpoints for progress and artifacts.
• poser-web ↔ poser-db for persistence.
• Both services ↔ Tigris for file storage.

CI workflow

• Feature branches: Open a PR to main to trigger lint + test jobs.
• Main branch: On merge, CI runs again and deploys to Fly.io.
• Checks: Ruff lint/format, analysis tests, backend tests, frontend tests.
• Deploy: GitHub Actions uses flyctl deploy with fly.web.toml and fly.analysis.toml.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• YOLOv11 by Ultralytics
• MediaPipe by Google
• BoT-SORT tracking algorithm
• Open source computer vision community