feat:Add ModelMetadata, allow string or token ID inputs, extend Embeddings #220
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WalkthroughIntroduces a new ModelMetadata schema and exposes it via OpenAIModelOut.metadata. Expands prompt/input types to accept strings or token ID arrays in multiple schemas. Extends OpenAIEmbeddingsIn with stop_token_ids and return_tokens_as_token_ids. All changes are within src/libs/DeepInfra/openapi.yaml. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant API as DeepInfra API
participant Model as Model Runtime
rect rgba(230,245,255,0.5)
note over Client,API: Completions/Model invocation with flexible prompt
Client->>API: POST /... (prompt: string | [int])
API->>Model: Normalize prompt (text or token IDs)
alt prompt is [int]
note right of API: Skip tokenization
else prompt is string
API->>Model: Tokenize text
end
Model-->>API: Generated output (text or token IDs per request)
API-->>Client: Response (may include metadata)
end
rect rgba(240,255,230,0.5)
note over Client,API: Embeddings with expanded inputs
Client->>API: POST /embeddings (input: string | [string], stop_token_ids?, return_tokens_as_token_ids?)
API->>Model: Compute embeddings with stop controls
Model-->>API: Embeddings (and optional token IDs)
API-->>Client: Embeddings response
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
coderabbitai
bot
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
src/libs/DeepInfra/openapi.yaml (1)
7678-7680: Make metadata explicitly nullable and document it.This keeps OpenAIModelOut backward-compatible when metadata is absent.
Apply:
metadata: - $ref: '#/components/schemas/ModelMetadata' + $ref: '#/components/schemas/ModelMetadata' + nullable: true + description: Optional model metadata (present for models where this information is available).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (12)
src/libs/DeepInfra/Generated/DeepInfra..JsonSerializerContext.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.DeepInfraClient.OpenaiCompletions.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.DeepInfraClient.OpenaiCompletions2.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.IDeepInfraClient.OpenaiCompletions.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.IDeepInfraClient.OpenaiCompletions2.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.JsonSerializerContextTypes.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.Models.ModelMetadata.Json.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.Models.ModelMetadata.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.Models.ModelMetadataPricing.Json.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.Models.ModelMetadataPricing.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.Models.OpenAICompletionsIn.g.csis excluded by!**/generated/**src/libs/DeepInfra/Generated/DeepInfra.Models.OpenAIModelOut.g.csis excluded by!**/generated/**
📒 Files selected for processing (1)
src/libs/DeepInfra/openapi.yaml(4 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Test / Build, test and publish
🔇 Additional comments (1)
src/libs/DeepInfra/openapi.yaml (1)
7436-7447: Harden stop_token_ids schema – addminimum: 0underitems,uniqueItems: true, and update its description to note that when bothstop(strings) andstop_token_idsare provided, generation will halt on the first matched token from either set. The OpenAPI v3.0.4 spec supports these keywords; please verify the server enforces this precedence.
Comment on lines
+6766
to
+6794
| ModelMetadata: | ||
| title: ModelMetadata | ||
| required: | ||
| - description | ||
| - context_length | ||
| - max_tokens | ||
| - pricing | ||
| - tags | ||
| type: object | ||
| properties: | ||
| description: | ||
| title: Description | ||
| type: string | ||
| context_length: | ||
| title: Context Length | ||
| type: integer | ||
| max_tokens: | ||
| title: Max Tokens | ||
| type: integer | ||
| pricing: | ||
| title: Pricing | ||
| type: object | ||
| additionalProperties: | ||
| type: number | ||
| tags: | ||
| title: Tags | ||
| type: array | ||
| items: | ||
| type: string |
There was a problem hiding this comment.
🛠️ Refactor suggestion
ModelMetadata: add minimums and tighten collections.
- context_length/max_tokens should be non-negative.
- pricing benefits from a brief units description.
- tags should avoid duplicates.
Apply:
ModelMetadata:
title: ModelMetadata
@@
context_length:
title: Context Length
type: integer
+ minimum: 0
max_tokens:
title: Max Tokens
type: integer
+ minimum: 0
pricing:
title: Pricing
type: object
+ description: Map of pricing metrics (e.g., "prompt", "completion") to price per token in USD.
additionalProperties:
type: number
tags:
title: Tags
type: array
+ uniqueItems: true
items:
type: string📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| ModelMetadata: | |
| title: ModelMetadata | |
| required: | |
| - description | |
| - context_length | |
| - max_tokens | |
| - pricing | |
| - tags | |
| type: object | |
| properties: | |
| description: | |
| title: Description | |
| type: string | |
| context_length: | |
| title: Context Length | |
| type: integer | |
| max_tokens: | |
| title: Max Tokens | |
| type: integer | |
| pricing: | |
| title: Pricing | |
| type: object | |
| additionalProperties: | |
| type: number | |
| tags: | |
| title: Tags | |
| type: array | |
| items: | |
| type: string | |
| ModelMetadata: | |
| title: ModelMetadata | |
| required: | |
| - description | |
| - context_length | |
| - max_tokens | |
| - pricing | |
| - tags | |
| type: object | |
| properties: | |
| description: | |
| title: Description | |
| type: string | |
| context_length: | |
| title: Context Length | |
| type: integer | |
| minimum: 0 | |
| max_tokens: | |
| title: Max Tokens | |
| type: integer | |
| minimum: 0 | |
| pricing: | |
| title: Pricing | |
| type: object | |
| description: Map of pricing metrics (e.g., "prompt", "completion") to price per token in USD. | |
| additionalProperties: | |
| type: number | |
| tags: | |
| title: Tags | |
| type: array | |
| uniqueItems: true | |
| items: | |
| type: string |
🤖 Prompt for AI Agents
In src/libs/DeepInfra/openapi.yaml around lines 6766 to 6794, the ModelMetadata
schema needs tightening: add minimum: 0 to both context_length and max_tokens to
enforce non-negative integers; add a brief description on the pricing property
and its additionalProperties to document the units (e.g., price per token or per
1k tokens) while keeping additionalProperties as number; and set tags to include
uniqueItems: true to prevent duplicate tag entries. Make these changes in-place
within the ModelMetadata properties block.
Comment on lines
+7322
to
7327
| anyOf: | ||
| - type: string | ||
| - type: array | ||
| items: | ||
| type: integer | ||
| description: input prompt - a single string is currently supported |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Prompt now accepts token IDs: update description and add basic validation.
Align the description with the new anyOf and add simple bounds to avoid negative token IDs and empty arrays.
Apply:
prompt:
title: Prompt
- anyOf:
+ anyOf:
- type: string
- type: array
- items:
- type: integer
- description: input prompt - a single string is currently supported
+ minItems: 1
+ items:
+ type: integer
+ minimum: 0
+ description: Input prompt as text or as an array of token IDs.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| anyOf: | |
| - type: string | |
| - type: array | |
| items: | |
| type: integer | |
| description: input prompt - a single string is currently supported | |
| prompt: | |
| title: Prompt | |
| anyOf: | |
| - type: string | |
| - type: array | |
| minItems: 1 | |
| items: | |
| type: integer | |
| minimum: 0 | |
| description: Input prompt as text or as an array of token IDs. |
🤖 Prompt for AI Agents
In src/libs/DeepInfra/openapi.yaml around lines 7322 to 7327, the schema now
allows either a string or an array of integers but the description and
validation are out of date; update the description to state the prompt may be a
text string or an array of token IDs, and add basic validation on the array form
by adding minItems: 1 to prevent empty arrays and minimum: 0 on the integer
items to forbid negative token IDs; also add minLength: 1 to the string type to
prevent empty strings.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit