-
Notifications
You must be signed in to change notification settings - Fork 3
docs: add separate OpenAI, Anyway, and Traceloop guides #74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| # Anyway Implementation Guide | ||
|
|
||
| Version: v1.0 | ||
| Date: March 6, 2026 | ||
| Scope: OpenCTO Anyway tracing integration only | ||
|
|
||
| ## Purpose | ||
|
|
||
| This guide documents the current Anyway integration in OpenCTO and how to operate it safely. | ||
|
|
||
| ## Current State | ||
|
|
||
| Anyway tracing is implemented in the cloudbot worker only. | ||
|
|
||
| - Worker: `opencto/opencto-cloudbot-worker` | ||
| - Core file: `opencto/opencto-cloudbot-worker/src/index.ts` | ||
| - Pattern: fail-open trace buffer that never blocks user responses | ||
|
|
||
| No Anyway integration is currently wired into: | ||
|
|
||
| - `opencto/opencto-api-worker` | ||
| - `opencto/opencto-dashboard` | ||
| - `opencto/mobile-app` | ||
| - `opencto/cto-orchestrator` | ||
|
|
||
| ## Architecture | ||
|
|
||
| ```text | ||
| Webhook Event (Telegram/Slack/Infobip) | ||
| | | ||
| v | ||
| OpenCTO CloudBot Worker | ||
| - span buffer per request | ||
| - flush at request completion | ||
| | | ||
| v | ||
| Anyway ingest endpoint | ||
| ``` | ||
|
|
||
| ## Implementation Details | ||
|
|
||
| ### Enablement Gate | ||
|
|
||
| Tracing is enabled only when both are true: | ||
|
|
||
| 1. `OPENCTO_ANYWAY_ENABLED=true` | ||
| 2. `OPENCTO_ANYWAY_API_KEY` is configured | ||
|
|
||
| Code location: | ||
|
|
||
| - `anywayEnabled(env)` in `opencto/opencto-cloudbot-worker/src/index.ts` | ||
|
|
||
| ### Trace Buffer | ||
|
|
||
| `AnywayTraceBuffer` handles: | ||
|
|
||
| - trace ID generation | ||
| - span start/end with status and attributes | ||
| - batched flush to ingest endpoint | ||
|
|
||
| Default ingest endpoint: | ||
|
|
||
| - `https://api.anyway.sh/v1/ingest` | ||
|
|
||
| Request payload shape: | ||
|
|
||
| - `{ traces: [{ trace_id, spans }], metrics: [] }` | ||
|
|
||
| ### Instrumented Workflow Segments | ||
|
|
||
| Cloudbot spans are created around: | ||
|
|
||
| - Telegram webhook handling | ||
| - Slack webhook handling | ||
| - Infobip webhook handling | ||
| - lexical and semantic memory retrieval | ||
| - OpenAI response generation | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| | Variable | Purpose | | ||
| | --- | --- | | ||
| | `OPENCTO_ANYWAY_ENABLED` | Global enable flag (`true`/`false`) | | ||
| | `OPENCTO_ANYWAY_API_KEY` | Bearer token for ingest auth | | ||
| | `OPENCTO_ANYWAY_ENDPOINT` | Optional custom ingest URL override | | ||
|
|
||
| ## Deployment Runbook | ||
|
|
||
| 1. Set secrets: | ||
|
|
||
| ```bash | ||
| wrangler secret put OPENCTO_ANYWAY_API_KEY | ||
| ``` | ||
|
|
||
| 2. Set vars in `wrangler.toml`: | ||
|
|
||
| ```toml | ||
| OPENCTO_ANYWAY_ENABLED = "true" | ||
| OPENCTO_ANYWAY_ENDPOINT = "https://api.anyway.sh/v1/ingest" | ||
| ``` | ||
|
|
||
| 3. Deploy worker: | ||
|
|
||
| ```bash | ||
| wrangler deploy | ||
| ``` | ||
|
|
||
| 4. Verify: | ||
| - `GET /health` confirms worker is up. | ||
| - Trigger a webhook event. | ||
| - Confirm traces arrive in Anyway. | ||
|
|
||
| ## Operational Notes | ||
|
|
||
| - Integration is explicitly fail-open; telemetry failures are swallowed by design. | ||
| - Do not place user secrets in span attributes. | ||
| - Treat phone numbers and user IDs as sensitive identifiers; hash/redact where possible before adding new attributes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # OpenAI Implementation Guide | ||
|
|
||
| Version: v1.0 | ||
| Date: March 6, 2026 | ||
| Scope: OpenCTO OpenAI integrations only | ||
|
|
||
| ## Purpose | ||
|
|
||
| This guide documents how OpenCTO currently uses OpenAI across product surfaces and background services. | ||
|
|
||
| ## Where OpenAI Is Used | ||
|
|
||
| 1. Realtime voice assistant sessions (dashboard + mobile via API Worker token minting). | ||
| 2. OpenAI operational proxy endpoints (`models`, `usage`) for tool-driven platform observability. | ||
| 3. Incident/autonomy workflows in orchestrator. | ||
| 4. Multi-channel cloudbot responses and embeddings. | ||
| 5. Voice backend chat completion endpoint. | ||
|
|
||
| ## Architecture | ||
|
|
||
| ```text | ||
| Client (Dashboard/Mobile/SDK) | ||
| | | ||
| v | ||
| OpenCTO API Worker | ||
| - POST /api/v1/realtime/token | ||
| - GET /api/v1/cto/openai/models | ||
| - GET /api/v1/cto/openai/usage | ||
| | | ||
| v | ||
| OpenAI API | ||
| ``` | ||
|
|
||
| ## Core Implementation Points | ||
|
|
||
| ### Realtime Token Minting | ||
|
|
||
| - Endpoint: `POST /api/v1/realtime/token` | ||
| - File: `opencto/opencto-api-worker/src/index.ts` | ||
| - Behavior: | ||
| - resolves key by workspace/user via BYOK (`provider=openai`) or `OPENAI_API_KEY` fallback | ||
| - calls `https://api.openai.com/v1/realtime/client_secrets` | ||
| - returns ephemeral `clientSecret` to browser/mobile | ||
|
|
||
| ### Realtime Session Client | ||
|
|
||
| - Files: | ||
| - `opencto/opencto-dashboard/src/lib/realtime/openaiAdapter.ts` | ||
| - `opencto/mobile-app/src/api/realtime.ts` | ||
| - Behavior: | ||
| - connects to `wss://api.openai.com/v1/realtime?model=<model>` | ||
| - uses subprotocol auth: `openai-insecure-api-key.<clientSecret>` | ||
| - streams PCM audio, receives transcript/audio deltas | ||
| - executes function tools via backend proxy routes | ||
|
|
||
| ### OpenAI Proxy Endpoints | ||
|
|
||
| - Routes: | ||
| - `GET /api/v1/cto/openai/models` | ||
| - `GET /api/v1/cto/openai/usage?start=<date>&end=<date>` | ||
| - File: `opencto/opencto-api-worker/src/index.ts` (`proxyOpenAI`) | ||
| - Used by dashboard tool execution in `opencto/opencto-dashboard/src/lib/realtime/shared.ts` | ||
|
|
||
| ### BYOK Key Management | ||
|
|
||
| - File: `opencto/opencto-api-worker/src/providerKeys.ts` | ||
| - Routes: | ||
| - `GET /api/v1/llm/keys` | ||
| - `PUT /api/v1/llm/keys/openai` | ||
| - `DELETE /api/v1/llm/keys/openai` | ||
| - Security: | ||
| - AES-GCM at-rest encryption | ||
| - per user + workspace + provider key scope | ||
| - only masked hints are returned | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| | Area | Variables | | ||
| | --- | --- | | ||
| | API Worker | `OPENAI_API_KEY`, `RATE_LIMIT_REALTIME_PER_MINUTE`, `RATE_LIMIT_CTO_OPENAI_PER_MINUTE` | | ||
| | Voice backend | `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `OPENAI_MODEL` | | ||
| | Orchestrator | `OPENAI_API_KEY`, `OPENCTO_OPENAI_MODEL`, `OPENCTO_AGENT_MODEL` | | ||
| | Cloudbot worker | `OPENAI_API_KEY`, `OPENCTO_AGENT_MODEL`, `OPENCTO_EMBED_MODEL` | | ||
|
|
||
| ## Operational Runbook | ||
|
|
||
| 1. Set `OPENAI_API_KEY` in API Worker (`wrangler secret put OPENAI_API_KEY`). | ||
| 2. For multi-tenant setups, save per-workspace key via `PUT /api/v1/llm/keys/openai`. | ||
| 3. Validate realtime path: | ||
| - call `/api/v1/realtime/token` | ||
| - confirm non-empty `clientSecret` | ||
| 4. Validate ops path: | ||
| - call `/api/v1/cto/openai/models` | ||
| - call `/api/v1/cto/openai/usage` | ||
| 5. Monitor 429 and 5xx rates on: | ||
| - `realtime_token` | ||
| - `cto_openai_proxy` | ||
|
|
||
| ## References (Chicago 17B) | ||
|
|
||
| OpenAI. “Realtime API.” *OpenAI API Documentation*. Accessed March 6, 2026. https://developers.openai.com/api/docs/guides/realtime. | ||
|
|
||
| OpenAI. “Text Generation.” *OpenAI API Documentation*. Accessed March 6, 2026. https://developers.openai.com/api/docs/guides/text. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| # Traceloop Implementation Guide | ||
|
|
||
| Version: v1.0 | ||
| Date: March 6, 2026 | ||
| Scope: OpenCTO Traceloop strategy and rollout | ||
|
|
||
| ## Purpose | ||
|
|
||
| This guide defines how to implement Traceloop in OpenCTO. As of this date, there is no direct Traceloop integration in the repository. | ||
|
|
||
| ## Current State | ||
|
|
||
| - No `traceloop` package/config usage found in OpenCTO product surfaces. | ||
| - Existing telemetry patterns are: | ||
| - custom trace context headers in legacy SDK paths | ||
| - Anyway tracing in cloudbot worker | ||
|
|
||
| This means Traceloop should be introduced as a new telemetry layer, not as an in-place replacement. | ||
|
|
||
| ## Rollout Goals | ||
|
|
||
| 1. End-to-end request tracing for realtime and agent workflows. | ||
| 2. LLM span observability (latency, error, model usage metadata). | ||
| 3. Correlation across API worker, orchestrator, cloudbot, and client-facing sessions. | ||
| 4. Safe telemetry posture (no secrets/PII leakage). | ||
|
|
||
| ## Recommended Rollout Phases | ||
|
|
||
| ### Phase 1: API Worker Core | ||
|
|
||
| Instrument: | ||
|
|
||
| - `POST /api/v1/realtime/token` | ||
| - `GET /api/v1/cto/openai/models` | ||
| - `GET /api/v1/cto/openai/usage` | ||
| - `GET /api/v1/cto/cloudflare/*` | ||
|
|
||
| Required attributes: | ||
|
|
||
| - `workspace.id` | ||
| - `route.name` | ||
| - `llm.vendor` | ||
| - `llm.model` (when applicable) | ||
| - `http.status_code` | ||
| - `error.type` | ||
|
|
||
| ### Phase 2: Agent Backends | ||
|
|
||
| Instrument: | ||
|
|
||
| - `opencto/cto-orchestrator` incident cycles and OpenAI triage calls | ||
| - `opencto/opencto-cloudbot-worker` RAG retrieval + response generation + webhook lifecycle | ||
|
|
||
| ### Phase 3: Client Correlation | ||
|
|
||
| - propagate trace context IDs from API worker into dashboard/mobile session state | ||
| - attach trace IDs to support/debug payloads (without exposing secrets) | ||
|
|
||
| ### Phase 4: Governance | ||
|
|
||
| - add CI checks for telemetry redaction rules | ||
| - enforce denylist for sensitive fields: | ||
| - `token`, `secret`, `authorization`, API keys, raw credentials | ||
|
|
||
| ## Suggested Environment Variables | ||
|
|
||
| | Variable | Purpose | | ||
| | --- | --- | | ||
| | `TRACELOOP_API_KEY` | Auth for Traceloop ingestion | | ||
| | `TRACELOOP_BASE_URL` | Optional endpoint override | | ||
| | `TRACELOOP_ENABLED` | Feature flag for gradual rollout | | ||
| | `TRACELOOP_SERVICE_NAME` | Service-level identity per component | | ||
| | `TRACELOOP_ENV` | Environment tag (`dev`, `staging`, `production`) | | ||
|
|
||
| ## Suggested Service Matrix | ||
|
|
||
| | Service | Start Phase | Priority | | ||
| | --- | --- | --- | | ||
| | `opencto-api-worker` | Phase 1 | Critical | | ||
| | `cto-orchestrator` | Phase 2 | High | | ||
| | `opencto-cloudbot-worker` | Phase 2 | High | | ||
| | `opencto-voice-backend` | Phase 2/3 | Medium | | ||
| | dashboard/mobile clients | Phase 3 | Medium | | ||
|
|
||
| ## Minimal Implementation Checklist | ||
|
|
||
| 1. Add Traceloop SDK/config to `opencto-api-worker`. | ||
| 2. Wrap OpenAI and Cloudflare proxy paths with spans. | ||
| 3. Add span context propagation headers for cross-service correlation. | ||
| 4. Add redaction middleware for trace attributes. | ||
| 5. Repeat for orchestrator and cloudbot. | ||
| 6. Build latency/error dashboards per workflow. | ||
|
|
||
| ## Validation Checklist | ||
|
|
||
| - Trace appears for every `/api/v1/realtime/token` request. | ||
| - OpenAI model and usage proxy calls produce linked spans. | ||
| - Failed upstream requests show structured error attributes. | ||
| - No secret-bearing fields appear in traces. | ||
| - End-to-end correlation from user action to backend spans is queryable. | ||
|
|
||
| ## References (Chicago 17B) | ||
|
|
||
| Traceloop. “Introduction.” *Traceloop Documentation*. Accessed March 6, 2026. https://www.traceloop.com/docs. | ||
|
|
||
| OpenTelemetry. “What Is OpenTelemetry?” *OpenTelemetry Documentation*. Accessed March 6, 2026. https://opentelemetry.io/docs/what-is-opentelemetry/. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The runbook currently says to call
/api/v1/cto/openai/usagewithout query parameters, but the API worker forwardsstartandenddirectly to OpenAI asstart_time/end_time(seeproxyOpenAIcall path inopencto/opencto-api-worker/src/index.ts), and the dashboard tool contract requires both dates (get_openai_usageinopencto/opencto-dashboard/src/lib/realtime/shared.ts). Following this step as written sends empty values and can return upstream errors instead of validating the ops path.Useful? React with 👍 / 👎.