services and packages split

[DiTo97]

Here is the unified Work Plan for the engineering team. This document covers the architectural migration, configuration changes, automation scripts, and deployment strategy required to transition the toolkit workspace into a polyglot monorepo.

---

Work Plan: Polyglot Monorepo Migration

1. Executive Summary

We are transitioning the toolkit workspace from a Python-only environment to a Polyglot Monorepo (Python + TypeScript). This enables the development of a React-based frontend (toolkit-web) that consumes the Python backend via an automatically generated, type-safe TypeScript SDK.

2. Target Repository Structure

The file system will be reorganized to distinguish between runnable services, shared packages, and deployment configuration.

toolkit/ (Root)
├── pyproject.toml           # Root uv workspace config
├── package.json             # NEW: Root Node workspace config
├── uv.lock
├── scripts/
│   └── export_openapi.py    # NEW: Extracts OpenAPI JSON from FastAPI
├── deployment/              # NEW: Container orchestration
│   ├── docker-compose.yml
│   ├── Dockerfile.server
│   └── Dockerfile.web
├── services/                # RENAMED (was 'apps')
│   ├── toolkit-server/      # (Python) FastAPI Backend (was 'toolkit-api')
│   └── toolkit-web/         # (React) Frontend (was 'toolkit-service')
└── packages/
    ├── toolkit-core/        # (Python) Shared Core Logic
    ├── toolkit-client-py/   # (Python) API Client
    └── toolkit-client-ts/   # (TS) NEW: Generated API Client

---

3. Structural & Configuration Changes

A. Root Configuration

File: pyproject.toml (Update)
Update members to reflect the rename to services and exclude Node directories.

[tool.uv]
members = [
    "services/toolkit-server",
    "packages/toolkit-core",
    "packages/toolkit-client-py"
]
exclude = [
    "**/node_modules", 
    "**/dist", 
    "services/toolkit-web", 
    "packages/toolkit-client-ts"
]

File: package.json (Create)
Initialize the Node.js workspace root.

{
  "name": "toolkit-monorepo",
  "private": true,
  "workspaces": [
    "services/toolkit-web",
    "packages/toolkit-client-ts"
  ],
  "scripts": {
    "generate-client": "openapi --input ./openapi.json --output ./packages/toolkit-client-ts/src/generated --client axios"
  },
  "devDependencies": {
    "openapi-typescript-codegen": "^0.25.0"
  }
}

B. Package Setup

Package: packages/toolkit-client-ts
Create this folder. It will house the auto-generated code.

• File: package.json

{
  "name": "@toolkit/client",
  "version": "0.0.1",
  "main": "./dist/index.js",
  "types": "./dist/index.d.ts",
  "exports": {
    ".": "./src/index.ts" 
  }
}

Service: services/toolkit-web

• Add dependency: "@toolkit/client": "*"

---

4. Automation & Code Generation

A. OpenAPI Export Script

File: scripts/export_openapi.py
Used to extract the JSON spec without running the server.

import json
import argparse
# Adjust import to match your actual server structure
from toolkit_server.main import app 

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--output", default="openapi.json")
    args = parser.parse_args()
    
    print("Extracting OpenAPI schema...")
    with open(args.output, "w") as f:
        json.dump(app.openapi(), f, indent=2)
    print(f"✅ Exported to {args.output}")

B. CI/CD Pipeline (GitHub Actions)

File: .github/workflows/generate-client.yml
Automatically regenerates the TS client when the backend changes.

name: Generate Client SDKs
on:
  push:
    branches: ["main"]
    paths: ["services/toolkit-server/**", "packages/toolkit-core/**"]
permissions:
  contents: write
jobs:
  generate-sdk:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with: { python-version: '3.11' }
          
      - name: Install Backend Deps
        run: pip install uv && uv sync
          
      - name: Export OpenAPI JSON
        run: |
          export PYTHONPATH=$PYTHONPATH:$(pwd)/services/toolkit-server/src:$(pwd)/packages/toolkit-core/src
          uv run python scripts/export_openapi.py
          
      - name: Set up Node
        uses: actions/setup-node@v4
        with: { node-version: '20' }

      - name: Install Generator
        run: npm install

      - name: Generate TypeScript Client
        run: npm run generate-client

      - name: Commit & Push changes
        uses: stefanzweifel/git-auto-commit-action@v5
        with:
          commit_message: "chore(sdk): auto-generate typescript client [skip ci]"
          file_pattern: 'packages/toolkit-client-ts/**'

---

5. Deployment Architecture

A. Backend Container

File: deployment/Dockerfile.server

FROM python:3.11-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv
WORKDIR /app

# Configs
COPY pyproject.toml uv.lock ./
COPY services/toolkit-server/pyproject.toml services/toolkit-server/
COPY packages/toolkit-core/pyproject.toml packages/toolkit-core/
COPY packages/toolkit-client-py/pyproject.toml packages/toolkit-client-py/

RUN uv sync --frozen

# Source
COPY services/toolkit-server/src services/toolkit-server/src
COPY packages/toolkit-core/src packages/toolkit-core/src
COPY packages/toolkit-client-py/src packages/toolkit-client-py/src

ENV PYTHONPATH=/app/services/toolkit-server/src:/app/packages/toolkit-core/src
CMD ["uv", "run", "uvicorn", "toolkit_server.main:app", "--host", "0.0.0.0", "--port", "8000"]

B. Web Container

File: deployment/Dockerfile.web

FROM node:20-slim as builder
WORKDIR /app
COPY package.json package-lock.json ./
COPY services/toolkit-web/package.json services/toolkit-web/
COPY packages/toolkit-client-ts/package.json packages/toolkit-client-ts/
RUN npm install

COPY services/toolkit-web services/toolkit-web
COPY packages/toolkit-client-ts packages/toolkit-client-ts

RUN npm run build --workspace=services/toolkit-web

FROM nginx:alpine
COPY --from=builder /app/services/toolkit-web/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

C. Orchestration

File: deployment/docker-compose.yml

services:
  server:
    build:
      context: ..
      dockerfile: deployment/Dockerfile.server
    ports: ["8000:8000"]
    volumes:
      - ../services/toolkit-server/src:/app/services/toolkit-server/src
      - ../packages/toolkit-core/src:/app/packages/toolkit-core/src
      
  web:
    build:
      context: ..
      dockerfile: deployment/Dockerfile.web
    ports: ["3000:80"]
    depends_on:
      - server

---

6. Definition of Done

1. Repo restructured with services/, packages/, and deployment/.
2. uv workspace syncs correctly without errors.
3. npm install at root installs deps for web and ts-client.
4. Running python scripts/export_openapi.py generates a valid JSON.
5. Running npm run generate-client populates packages/toolkit-client-ts.
6. The React app can import import { X } from '@toolkit/client' and IntelliSense works.
7. docker-compose up --build launches both services successfully.