feat:Add optional prompt_cache_key to chat/completions OpenAPI schema #225
Conversation
WalkthroughAdded an optional string field prompt_cache_key (nullable) to the OpenAPI input schema for chat/completions in src/libs/DeepInfra/openapi.yaml. No required properties changed and no other schema or control-flow modifications indicated. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Client
participant API as DeepInfra API
participant Cache as Prompt Cache
participant Model as Model Runtime
Client->>API: POST /chat/completions {prompt, prompt_cache_key?}
alt prompt_cache_key provided
API->>Cache: Lookup(key)
alt Cache hit
Cache-->>API: Cached prompt/context
else Cache miss
API->>Model: Generate with prompt
Model-->>API: Output
API->>Cache: Store(key, prompt/context)
end
else no key
API->>Model: Generate with prompt
Model-->>API: Output
end
API-->>Client: Response
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Pre-merge checks (1 passed, 1 warning, 1 inconclusive)❌ Failed Checks (1 warning, 1 inconclusive)
✅ Passed Checks (1 passed)
Poem
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
coderabbitai
bot
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
src/libs/DeepInfra/openapi.yaml (3)
7290-7294: Clarify semantics: scope, TTL, and precedence vs message-level cache_controlPlease document:
- Scope (per user/account/team) and cross-tenant isolation
- TTL/eviction behavior
- Precedence with ChatCompletion[User|System|Assistant|Tool]Message.cache_control if both are set
7290-7294: Parity: consider adding prompt_cache_key to OpenAICompletionsInIf caching is supported for non-chat completions too, expose the same field there for consistency.
YAML patch (outside this hunk, under components.schemas.OpenAICompletionsIn.properties):
+ prompt_cache_key: + title: Prompt Cache Key + type: string + description: 'A key to identify the prompt cache for reuse across requests. Scoped to the authenticated account/team. Case-sensitive.' + minLength: 1 + maxLength: 256 + pattern: '^[A-Za-z0-9._:-]+$' + nullable: true
7290-7294: Wording nit in description“identify prompt cache” → “identify the prompt cache”.
- description: 'A key to identify prompt cache for reuse across requests. If provided, the prompt will be cached and can be reused in subsequent requests with the same key.' + description: 'A key to identify the prompt cache for reuse across requests. If provided, the prompt will be cached and can be reused in subsequent requests with the same key.'
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (5)
src/libs/DeepInfra/Generated/DeepInfra.DeepInfraClient.OpenaiChatCompletions.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.DeepInfraClient.OpenaiChatCompletions2.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.IDeepInfraClient.OpenaiChatCompletions.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.IDeepInfraClient.OpenaiChatCompletions2.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.Models.OpenAIChatCompletionsIn.g.csis excluded by!**/generated/**
📒 Files selected for processing (1)
src/libs/DeepInfra/openapi.yaml(1 hunks)
🔇 Additional comments (1)
src/libs/DeepInfra/openapi.yaml (1)
7290-7294: LGTM: non-breaking optional field addedOptional, nullable string fits existing schema style. Safe addition.
| prompt_cache_key: | ||
| title: Prompt Cache Key | ||
| type: string | ||
| description: 'A key to identify prompt cache for reuse across requests. If provided, the prompt will be cached and can be reused in subsequent requests with the same key.' | ||
| nullable: true |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add length/charset constraints to prevent abuse and collisions
Constrain the key to a sane length and allowed chars; avoids accidental PII, log injection, and oversized headers/bodies.
prompt_cache_key:
title: Prompt Cache Key
type: string
- description: 'A key to identify prompt cache for reuse across requests. If provided, the prompt will be cached and can be reused in subsequent requests with the same key.'
+ description: 'A key to identify the prompt cache for reuse across requests. Scoped to the authenticated account/team. Case-sensitive.'
+ minLength: 1
+ maxLength: 256
+ pattern: '^[A-Za-z0-9._:-]+$'
nullable: true📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| prompt_cache_key: | |
| title: Prompt Cache Key | |
| type: string | |
| description: 'A key to identify prompt cache for reuse across requests. If provided, the prompt will be cached and can be reused in subsequent requests with the same key.' | |
| nullable: true | |
| prompt_cache_key: | |
| title: Prompt Cache Key | |
| type: string | |
| description: 'A key to identify the prompt cache for reuse across requests. Scoped to the authenticated account/team. Case-sensitive.' | |
| minLength: 1 | |
| maxLength: 256 | |
| pattern: '^[A-Za-z0-9._:-]+$' | |
| nullable: true |
🤖 Prompt for AI Agents
In src/libs/DeepInfra/openapi.yaml around lines 7290-7294, the prompt_cache_key
schema lacks length and charset constraints; add validation to limit size and
allowed characters to prevent PII, log injection, and oversized payloads. Update
the schema to include minLength (e.g. 1), maxLength (e.g. 128), and a
restrictive pattern that only permits safe characters (for example alphanumeric
and a small set of separators like _ - . :), and adjust the description to note
these limits; ensure the regex disallows spaces and control characters.
Summary by CodeRabbit
New Features
Documentation