fix: change logprobs type from bool to int in Completions endpoint#5343
fix: change logprobs type from bool to int in Completions endpoint#5343extrasmall0 wants to merge 1 commit intollamastack:mainfrom
Conversation
The OpenAI Completions API expects logprobs as an integer (0-5) specifying how many top log probabilities to return per token. The Chat Completions endpoint correctly uses bool, but the legacy Completions model had the wrong type, causing 400 errors when passing an integer value. Fixes llamastack#5253
extrasmall0
requested review from
ashwinb,
bbrowning,
cdoern,
ehhuang,
franciscojavierarceo,
leseb,
mattf and
raghotham
as code owners
|
Hi @extrasmall0! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
1 participant
Fixes #5253
The
logprobsfield inOpenAICompletionRequestWithExtraBodywas typed asboolbut the OpenAI Completions API expects an integer (0–5) indicating how many top log probabilities to include per output token. This caused a 400 error when callers passed an int:Changes:
models.py:logprobs: bool | None→logprobs: int | Nonewithge=0, le=5llama-stack-spec.yaml: matching schema update (boolean→integerwith min/max)The Chat Completions model (
OpenAIChatCompletionRequestWithExtraBody) is unchanged — itslogprobsis correctly a boolean there.