Skip to content

feat(model-routing): add openai models, full claude catalog, passthrough routing#35

Merged
RustMunkey merged 2 commits intomainfrom
feat/openai-models
Mar 14, 2026
Merged

feat(model-routing): add openai models, full claude catalog, passthrough routing#35
RustMunkey merged 2 commits intomainfrom
feat/openai-models

Conversation

@RustMunkey
Copy link
Copy Markdown
Owner

@RustMunkey RustMunkey commented Mar 14, 2026

Summary by CodeRabbit

  • New Features

    • Support for OpenAI models (GPT-5, O-series) alongside Claude and Ollama
    • Passthrough for non-catalog models via provider-prefix inference
    • Model selection resolves by plan tier with automatic fallback
    • Added OpenAI API key configuration
  • Bug Fixes

    • Updated plan tier allowances and default model assignments
    • Adjusted billing multipliers (unknown/passthrough models default to 2x)

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 14, 2026

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: 33f49ab9-ad9f-4b3f-8c03-2dd543413e2b

📥 Commits

Reviewing files that changed from the base of the PR and between 92db39a and a2a8107.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml
  • services/runtime/tests/test_runner_routing.py

📝 Walkthrough

Walkthrough

Adds OpenAI model support: expands the model catalog and provider inference, introduces OpenAIRunner and OpenAI API key config, updates runtime routing to dispatch by model-prefix (Ollama/OpenAI/Anthropic), and adjusts multipliers, passthrough, and model-resolution logic.

Changes

Cohort / File(s) Summary
Model Catalog Core
packages/model/src/catalog.ts
Expanded ModelDef provider to include openai, added deprecated?, added many OpenAI/Claude entries, changed DEFAULT_MODEL.m1, added inferProvider(), resolveModel(), ModelAccessResult.passthrough?, changed unknown-model multiplier default to 2, and updated validateModelAccess() for passthrough logic.
Model Catalog Tests
packages/model/src/catalog.test.ts
Updated tests for getModel, getModelMultiplier, validateModelAccess, getAllowedModels, resolveModel; added inferProvider tests and new expectations for claude/gpt/o3/o4 IDs and passthrough behavior.
OpenAI Runtime
services/runtime/src/openai_runner.py
New OpenAIRunner implementing async chat loop (run() / _run_loop()), token accounting, MAX_TURNS, timeout handling, and RunResult structure for OpenAI models.
Runtime Routing & Billing
services/runtime/src/runner.py
Routing now dispatches by prefix: ollama/* → OllamaRunner, gpt-*/o* → OpenAIRunner (requires OpenAI key), claude-* → Anthropic (requires Anthropic key); added OpenAI prefixes to multipliers, introduced passthrough multiplier constant, and updated multiplier lookup.
Runtime Configuration
services/runtime/src/config.py
Added openai_api_key: str = "" to Settings for OpenAI credential storage.
CI Workflow
.github/workflows/ci.yml
Added three packages (@maschina/webhooks, @maschina/search, @maschina/compliance) to TS build/test dependency sequence.
Runtime Tests
services/runtime/tests/test_runner_routing.py
Updated routing/multiplier tests to include gpt-5-mini, gpt-5, o3 multipliers and changed unknown-model fallback to passthrough multiplier 2x.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant Router as Runtime Router
    participant Config as Settings
    participant OAI as OpenAI API
    participant OAIRunner as OpenAI Runner
    participant Anthropic as Anthropic API
    participant Ollama as Ollama Runner

    User->>Router: request(model_id, input)
    Router->>Router: infer provider from model_id
    alt provider == openai
        Router->>Config: read openai_api_key
        Config-->>Router: api_key
        Router->>OAIRunner: instantiate(model_id, api_key)
        Router->>OAIRunner: run(input)
        OAIRunner->>OAI: create_chat_completion(messages)
        OAI-->>OAIRunner: response (tokens, finish_reason)
        OAIRunner-->>Router: RunResult(output, tokens, turns)
    else provider == anthropic
        Router->>Anthropic: forward request
    else provider == ollama
        Router->>Ollama: forward request
    else
        Router-->>User: RuntimeError (unknown provider / missing key)
    end
    Router-->>User: response + billing metadata
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 A hop, a sniff, a model named right,
GPT and Claude join in daylight.
I infer prefixes, route with care,
Passthrough and multipliers dance in the air.
Small rabbit cheers—new runners take flight!

📝 Coding Plan
  • Generate coding plan for human review comments

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

Comment @coderabbitai help to get the list of available commands and usage tips.

@RustMunkey RustMunkey merged commit b3f389f into main Mar 14, 2026
26 checks passed
@RustMunkey RustMunkey deleted the feat/openai-models branch March 14, 2026 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant