[BUG]LLM responce validation error

[yjing86]

open_llm_service.py generate_completion function uses
response = self.client.beta.chat.completions.parse( model=self.settings.llm_model, messages=[{"role": "user", "content": prompt}], response_format=response_model, )
to validate LLM's response. However, this doesn't work and logger info is as below:
WARNING:app.services.llm_service:LLM returned None response INFO: 127.0.0.1:50854 - "POST /api/v1/query HTTP/1.1" 200 OK ^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_base_client.py", line 1063, in _request return self._process_response( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_base_client.py", line 1162, in _process_response return api_response.parse() ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_response.py", line 319, in parse parsed = self._options.post_parser(parsed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/resources/beta/chat/completions.py", line 154, in parser return _parse_chat_completion( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 110, in parse_chat_completion "parsed": maybe_parse_content( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 161, in maybe_parse_content return _parse_content(response_format, message.content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 221, in _parse_content return cast(ResponseFormatT, model_parse_json(response_format, content)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_compat.py", line 169, in model_parse_json return model.model_validate_json(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/pydantic/main.py", line 656, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for StrArrayResponseModel Input should be an object [type=model_type, input_value=['quantum信息', '离散...', '计算机图形学'], input_type=list] For further information visit https://errors.pydantic.dev/2.10/v/model_type INFO:app.services.llm.openai_llm_service:generate_completion execution completed. INFO:app.services.llm_service:Raw response from LLM: None WARNING:app.services.llm_service:LLM returned None response INFO: 127.0.0.1:50854 - "POST /api/v1/query HTTP/1.1" 200 OK return api_response.parse() ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_response.py", line 319, in parse parsed = self._options.post_parser(parsed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/resources/beta/chat/completions.py", line 154, in parser return _parse_chat_completion( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 110, in parse_chat_completion "parsed": maybe_parse_content( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 161, in maybe_parse_content return _parse_content(response_format, message.content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 221, in _parse_content return cast(ResponseFormatT, model_parse_json(response_format, content)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_compat.py", line 169, in model_parse_json return model.model_validate_json(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/pydantic/main.py", line 656, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for StrArrayResponseModel Input should be an object [type=model_type, input_value=['quantum信息', '离散...', '计算机图形学'], input_type=list] For further information visit https://errors.pydantic.dev/2.10/v/model_type

I'm using Owen online model.

[tomsmoker]

Hey @yjing86, are you trying to use a non-OpenAI model with the OpenAI model code? Can you show me where you add your model environment variable?

[lwing1966]

open_llm_service.py generate_completion function uses response = self.client.beta.chat.completions.parse( model=self.settings.llm_model, messages=[{"role": "user", "content": prompt}], response_format=response_model, ) to validate LLM's response. However, this doesn't work and logger info is as below: WARNING:app.services.llm_service:LLM returned None response INFO: 127.0.0.1:50854 - "POST /api/v1/query HTTP/1.1" 200 OK ^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_base_client.py", line 1063, in _request return self._process_response( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_base_client.py", line 1162, in _process_response return api_response.parse() ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_response.py", line 319, in parse parsed = self._options.post_parser(parsed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/resources/beta/chat/completions.py", line 154, in parser return _parse_chat_completion( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 110, in parse_chat_completion "parsed": maybe_parse_content( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 161, in maybe_parse_content return _parse_content(response_format, message.content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 221, in _parse_content return cast(ResponseFormatT, model_parse_json(response_format, content)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_compat.py", line 169, in model_parse_json return model.model_validate_json(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/pydantic/main.py", line 656, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for StrArrayResponseModel Input should be an object [type=model_type, input_value=['quantum信息', '离散...', '计算机图形学'], input_type=list] For further information visit https://errors.pydantic.dev/2.10/v/model_type INFO:app.services.llm.openai_llm_service:generate_completion execution completed. INFO:app.services.llm_service:Raw response from LLM: None WARNING:app.services.llm_service:LLM returned None response INFO: 127.0.0.1:50854 - "POST /api/v1/query HTTP/1.1" 200 OK return api_response.parse() ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_response.py", line 319, in parse parsed = self._options.post_parser(parsed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/resources/beta/chat/completions.py", line 154, in parser return _parse_chat_completion( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 110, in parse_chat_completion "parsed": maybe_parse_content( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 161, in maybe_parse_content return _parse_content(response_format, message.content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 221, in _parse_content return cast(ResponseFormatT, model_parse_json(response_format, content)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_compat.py", line 169, in model_parse_json return model.model_validate_json(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/pydantic/main.py", line 656, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for StrArrayResponseModel Input should be an object [type=model_type, input_value=['quantum信息', '离散...', '计算机图形学'], input_type=list] For further information visit https://errors.pydantic.dev/2.10/v/model_type

I'm using Owen online model.

you must included " the response must be this json format :
{'answer':['response']} " in the end of the query of the column.it's ok.

because the local llm response not include {'answer':.....}

[yjing86]

Hey @yjing86, are you trying to use a non-OpenAI model with the OpenAI model code? Can you show me where you add your model environment variable?

Yep, Im using qwen and local models reasoned by VLLM.
.env file:

DIMENSIONS=1024
EMBEDDING_PROVIDER=openai
EMBEDDING_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
EMBEDDING_MODEL=text-embedding-v3
LLM_PROVIDER=openai
LLM_MODEL= qwen-max
OPENAI_API_KEY=sk-a82bdc5e4bb04c0097a6be7e85887c90
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1

and I also modified openai_embedding_service.py file (

  self.client = OpenAI(
            api_key=settings.openai_api_key,
            base_url=self.settings.embedding_base_url,
        )

) and openai_llm_service.py file (

"""OpenAI completion service implementation."""

import logging
from typing import Any, Optional, Type, List

from openai import OpenAI
from pydantic import BaseModel

from app.core.config import Settings
from app.services.llm.base import CompletionService

logger = logging.getLogger(__name__)


class OpenAICompletionService(CompletionService):
    """OpenAI completion service implementation."""

    def __init__(self, settings: Settings) -> None:
        self.settings = settings
        if settings.openai_api_key:
            self.client = OpenAI(api_key=settings.openai_api_key,
                                 base_url=settings.llm_base_url)
        else:
            self.client = None  # type: ignore
            logger.warning(
                "OpenAI API key is not set. LLM features will be disabled."
            )

    async def get_embeddings(self, texts: List[str]) -> List[List[float]]:
        """Get embeddings for text."""
        if self.client is None:
            logger.warning(
                "OpenAI client is not initialized. Skipping embeddings."
            )
            return []
        return [
            embedding.embedding
            for embedding in self.client.embeddings.create(
                input=texts, model=self.model
            ).data
        ]

    async def generate_completion(
            self, prompt: str, response_model: Type[BaseModel]
    ) -> Optional[BaseModel]:
        """Generate a completion from the language model."""

        logger.info("Starting generate_completion...")
        logger.info(f"Prompt provided: {prompt}")
        logger.info(f"Response model type: {response_model}")

        try:
            logger.info(f"Response model schema: {response_model.model_json_schema()}")
        except AttributeError as schema_error:
            logger.warning(f"Unable to retrieve model schema: {schema_error}")

        if self.client is None:
            logger.error("OpenAI client is not initialized. Cannot proceed with generation.")
            return None

        logger.info(f"Using OpenAI model: {self.settings.llm_model}")

        try:

            # raw_response = self.client.chat.completions.create(
            #     model=self.settings.llm_model,
            #     messages=[{"role": "user", "content": prompt}],
            # )
            # logger.info(f"Raw response received: {raw_response}")
            prompt="Please return your answer in JSON format."+prompt
            response = self.client.beta.chat.completions.parse(
                model=self.settings.llm_model,
                messages=[{"role": "user", "content": prompt}],
                response_format=response_model,
            )

            logger.info("Raw response received from OpenAI API.")
            logger.debug(f"Full response: {response}")

            parsed_response = response.choices[0].message.parsed
            logger.info(f"Parsed response: {parsed_response}")

            if parsed_response is None:
                logger.warning("Parsed response is None. No valid output from OpenAI API.")
                return None

            # Validate the response against the model
            try:
                validated_response = response_model(**parsed_response.model_dump())
                logger.info("Response successfully validated against response_model.")
                logger.debug(f"Validated response content: {validated_response.model_dump()}")

                if all(value is None for value in validated_response.model_dump().values()):
                    logger.warning("All fields in the validated response are None.")
                    return None

                return validated_response
            except ValueError as validation_error:
                logger.error(f"Validation error during response model parsing: {validation_error}")
                logger.debug(f"Invalid response content: {parsed_response}")
                return None

        except Exception as api_error:
            logger.error(f"Error occurred during API request: {api_error}", exc_info=True)
            logger.debug(f"Prompt that caused the error: {prompt}")
            return None
        finally:
            logger.info("generate_completion execution completed.")

    async def decompose_query(self, query: str) -> dict[str, Any]:
        """Decompose the query into smaller sub-queries."""
        if self.client is None:
            logger.warning(
                "OpenAI client is not initialized. Skipping decomposition."
            )
            return {"sub_queries": [query]}

        # TODO: Implement the actual decomposition logic here
        return {"sub_queries": [query]}
) 

to support local models. Actually I didn't find which code includes " the response must be this json format : {'answer':['response']} ", which I didn't add by myself either. Modifying llm_reponses.py does help(generated by chatgpt).

"""Pydantic models for validating responses from the LLM API."""

import logging
from typing import Any, List, Optional

from pydantic import BaseModel, Field, ValidationInfo, field_validator, model_validator

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)



class BaseResponseModel(BaseModel):
    """Base class for response models with common validation logic."""

    @classmethod
    def validate_none(cls, v: Any) -> Optional[Any]:
        """Validate if the value is None or "none"."""
        if v is None or (
            isinstance(v, str)
            and v.lower() in ["none", "not found", "null", ""]
        ):
            return None
        return v

class ArrayResponseModel(BaseResponseModel):
    """Base class for array response models."""

    @classmethod
    def validate_array(cls, v: Any, max_length: Optional[int] = None) -> Optional[List[Any]]:
        """Validate if the value is a list or None."""
        v = cls.validate_none(v)
        if v is None:
            return None
        if len(v) == 1 and v[0] == "None":
            return None
        if not isinstance(v, list):
            raise ValueError("Must be a list or None")
        if max_length and len(v) > max_length:
            logger.error(f"Length of response is greater than {max_length}")
            return v[:max_length]
        return v


# -----------------------------------------------------------------------------
# 1. BoolResponseModel
# -----------------------------------------------------------------------------
class BoolResponseModel(BaseResponseModel):
    """Pydantic model for validating boolean responses."""

    answer: Optional[bool] = Field(description="The boolean answer to the query")

    @model_validator(mode="before")
    @classmethod
    def fix_input_if_bool(cls, data):
        """
        若最外层是布尔值/字符串/列表，就封装成 { "answer": ... }
        """
        if isinstance(data, list):
            logger.info("Detected top-level list for BoolResponseModel, converting to {answer: ...}.")
            if len(data) == 1:
                return {"answer": data[0]}
            # 多元素列表时的策略，可自行决定
            return {"answer": any(data)}  # 比如自定义逻辑
        if isinstance(data, str) and data.strip().lower() == "none":
            logger.info("Detected top-level string 'None' for BoolResponseModel => converting to {answer: None}.")
            return {"answer": None}
        if isinstance(data, str) and data.lower() in ["true", "false"]:
            logger.info("Detected top-level string for BoolResponseModel, converting to {answer: bool}.")
            return {"answer": data}

        if isinstance(data, bool):
            logger.info("Detected top-level bool for BoolResponseModel, converting to {answer: bool}.")
            return {"answer": data}

        # 否则原样返回（可能是 { "answer": bool } 本身）
        return data

    @field_validator("answer", mode="before")
    def validate_bool(cls, v: Any) -> Optional[bool]:
        """Validate if the value is a boolean or None."""
        v = cls.validate_none(v)
        if v is None:
            return None
        if isinstance(v, str):
            if v.lower() == "true":
                return True
            elif v.lower() == "false":
                return False
        elif isinstance(v, bool):
            return v
        raise ValueError("Must be a boolean value or None")

# -----------------------------------------------------------------------------
# 2. IntResponseModel
# -----------------------------------------------------------------------------
class IntResponseModel(BaseResponseModel):
    """Pydantic model for validating integer responses."""

    answer: Optional[int] = Field(description="The integer answer to the query")

    @model_validator(mode="before")
    @classmethod
    def fix_input_if_list(cls, data):
        # 1) 如果是数组
        if isinstance(data, list):
            logger.info("Detected top-level list for IntResponseModel, converting to {answer: ...}.")
            if len(data) == 1:
                return {"answer": data[0]}
            return {"answer": data}

        # 2) 如果是整型或浮点数
        if isinstance(data, (int, float)):
            logger.info("Detected top-level number for IntResponseModel, converting to {answer: ...}.")
            return {"answer": data}

        # 3) 如果是字符串（也可做特殊处理）
        if isinstance(data, str):
            logger.info("Detected top-level string for IntResponseModel, converting to {answer: ...}.")
            return {"answer": data}

        # 否则保持原样
        return data

    @field_validator("answer", mode="before")
    def validate_int(cls, v: Any) -> Optional[int]:
        """Validate if the value is an integer or None."""
        v = cls.validate_none(v)
        if v is None:
            return None
        if isinstance(v, list) and len(v) == 1:
            v = v[0]
        if isinstance(v, float):
            raise ValueError("Must be an integer value, not a float")
        try:
            return int(v)
        except ValueError:
            raise ValueError("Must be an integer value or None")

# -----------------------------------------------------------------------------
# 3. IntArrayResponseModel
# -----------------------------------------------------------------------------
class IntArrayResponseModel(ArrayResponseModel):
    """Pydantic model for validating integer array responses."""

    answer: Optional[List[int]] = Field(description="The list of integer answers to the query")

    @model_validator(mode="before")
    @classmethod
    def fix_input_if_list(cls, data):
        # 情况1：最外层是 list
        if isinstance(data, list):
            logger.info("Detected top-level list for IntArrayResponseModel, converting to {answer: list}.")
            return {"answer": data}

        # 情况2：最外层是单个 int 或 float => 先变成列表
        if isinstance(data, (int, float)):
            logger.info("Detected top-level number for IntArrayResponseModel, converting to {answer: [number]}.")
            return {"answer": [data]}  # 注意，这里是 [data]，因为是数组

        # 情况3：最外层是字符串 => 可能需要转为 int？ 或者报错/自行决定
        if isinstance(data, str):
            logger.info("Detected top-level string for IntArrayResponseModel, converting to {answer: [str->int?]} or [str].")
            # 如果你想尝试 parse int:
            try:
                num = int(data)
                return {"answer": [num]}
            except ValueError:
                # 如果无法转int，看你要不要直接放进列表
                return {"answer": [data]}

        # 否则保持原样
        return data

    @field_validator("answer", mode="before")
    def validate_int_array(cls, v: Any, info: ValidationInfo) -> Optional[List[int]]:
        int_rule = info.data.get("int_rule")
        max_length = int_rule.length if int_rule else None
        v = cls.validate_array(v, max_length)
        if v is None:
            return None
        if isinstance(v, (int, float)):
            v = [v]
        try:
            return [int(item) for item in v]
        except ValueError:
            raise ValueError("All items must be valid integers")

# -----------------------------------------------------------------------------
# 4. StrArrayResponseModel
# -----------------------------------------------------------------------------
class StrArrayResponseModel(ArrayResponseModel):
    """
    Pydantic model for validating string array responses,
    but the model expects { "answer": [...] }.
    """

    answer: Optional[List[str]] = Field(
        description="The list of string answers to the query"
    )

    # 1) 先整体预处理：若输入本身是一个数组，就封装成 { "answer": ... }
    @model_validator(mode="before")
    @classmethod
    def fix_input_if_list(cls, data):
        """
        在字段验证前，对根输入进行检查：
        如果 data 是 list，就封装成 { "answer": data }。
        """
        if isinstance(data, list):
            logger.info("Detected top-level list, converting to {answer: list}.")
            return {"answer": data}
        return data

    # 2) 再针对 answer 字段做校验
    @field_validator("answer", mode="before")
    @classmethod
    def validate_str_array(
        cls, v: Any, info: ValidationInfo
    ) -> Optional[List[str]]:
        """Validate if the value is a string array or None."""
        # 这里可读到 str_rule、int_rule 等
        str_rule = info.data.get("str_rule")
        int_rule = info.data.get("int_rule")
        max_length = int_rule.length if int_rule else None

        v = cls.validate_array(v, max_length)
        if v is None:
            return None
        if isinstance(v, str):
            v = [v]

        if not all(isinstance(item, str) for item in v):
            raise ValueError("All items must be strings")

        if str_rule and str_rule.type == "must_return" and str_rule.options:
            return [i for i in v if i in str_rule.options]
        return v


# -----------------------------------------------------------------------------
# 5. StrResponseModel
# -----------------------------------------------------------------------------
class StrResponseModel(BaseResponseModel):
    """Pydantic model for validating string responses."""

    answer: Optional[str] = Field(description="The string answer to the query")

    @model_validator(mode="before")
    @classmethod
    def fix_input_if_str(cls, data):
        """
        在字段验证前，对根输入进行检查和适配：
        - 如果输入是列表且只有一个元素，提取该元素。
        - 如果输入是非字符串类型（如整数、布尔值等），尝试转换为字符串。
        - 如果输入是字符串，直接封装为 { "answer": data }。
        """
        if isinstance(data, list):
            if len(data) == 1:
                logger.info("Detected top-level single-element list, converting to {answer: element}.")
                return {"answer": data[0]}
            else:
                logger.info("Detected top-level multi-element list, joining into a single string.")
                return {"answer": " ".join(str(item) for item in data)}

        if isinstance(data, (int, float, bool)):
            logger.info("Detected top-level number/bool, converting to string in {answer: ...}.")
            return {"answer": str(data)}

        if isinstance(data, str):
            logger.info("Detected top-level string, converting to {answer: data}.")
            return {"answer": data}

        # 默认返回原始数据
        return data

    @field_validator("answer", mode="before")
    @classmethod
    def validate_str(cls, v: Any, info: ValidationInfo) -> Optional[str]:
        """Validate if the value is a string or None."""
        v = cls.validate_none(v)  # 先检查是否是 None 或 "null"
        if v is None:
            return None

        if isinstance(v, list) and len(v) == 1:
            v = v[0]

        if not isinstance(v, str):
            raise ValueError("Must be a string")

        str_rule = info.data.get("str_rule")
        if (
            str_rule
            and str_rule.type == "must_return"
            and str_rule.options is not None
            and v not in str_rule.options
        ):
            return None

        return v
# -----------------------------------------------------------------------------
# 6. KeywordsResponseModel
# -----------------------------------------------------------------------------
class KeywordsResponseModel(ArrayResponseModel):
    """Pydantic model for validating keyword responses."""
    keywords: Optional[List[str]] = Field(description="The extracted keywords from the query")

    @model_validator(mode="before")
    @classmethod
    def fix_input_if_list(cls, data):
        if isinstance(data, list):
            logger.info("Detected top-level list for KeywordsResponseModel, converting to {keywords: ...}.")
            return {"keywords": data}

        # 如果返回的是单个字符串 => 强行包进 list
        if isinstance(data, str):
            logger.info("Detected top-level string for KeywordsResponseModel => converting to single-element list.")
            return {"keywords": [data]}

        # 如果是 int/float/bool => 看你要不要加处理
        if isinstance(data, (int, float, bool)):
            logger.info("Detected top-level number/bool for KeywordsResponseModel => convert to string in a list ?")
            return {"keywords": [str(data)]}

        return data

    @field_validator("keywords", mode="before")
    def validate_keywords(cls, v: Any) -> Optional[List[str]]:
        return cls.validate_array(v)

# -----------------------------------------------------------------------------
# 7. SubQueriesResponseModel
# -----------------------------------------------------------------------------
class SubQueriesResponseModel(ArrayResponseModel):
    sub_queries: Optional[List[str]] = Field(description="The decomposed sub-queries")

    @model_validator(mode="before")
    @classmethod
    def fix_input_if_list(cls, data):
        if isinstance(data, list):
            logger.info("Detected top-level list for SubQueriesResponseModel, converting to {sub_queries: ...}.")
            return {"sub_queries": data}

        if isinstance(data, str):
            logger.info("Detected top-level string => convert to single-element list for SubQueriesResponseModel.")
            return {"sub_queries": [data]}

        # 是否要处理 int/float/bool
        if isinstance(data, (int, float, bool)):
            logger.info("Detected top-level number/bool => convert to str in single-element list for SubQueriesResponseModel.")
            return {"sub_queries": [str(data)]}

        return data

    @field_validator("sub_queries", mode="before")
    def validate_sub_queries(cls, v: Any) -> Optional[List[str]]:
        return cls.validate_array(v)


# -----------------------------------------------------------------------------
# 8. SchemaRelationship & SchemaResponseModel
# -----------------------------------------------------------------------------
class SchemaRelationship(BaseModel):
    """Pydantic model for a schema relationship."""
    head: str = Field(description="The head entity of the relationship")
    relation: str = Field(description="The relation between the head and tail entities")
    tail: str = Field(description="The tail entity of the relationship")

class SchemaResponseModel(ArrayResponseModel):
    relationships: Optional[List[SchemaRelationship]] = Field(description="The relationships in the schema")

    @model_validator(mode="before")
    @classmethod
    def fix_input_if_list(cls, data):
        if isinstance(data, list):
            logger.info("Detected top-level list for SchemaResponseModel, converting to {relationships: ...}.")
            return {"relationships": data}

        # 如果是单个 dict => 变成列表
        if isinstance(data, dict):
            logger.info("Detected top-level single dict => making it a single-element relationships list.")
            return {"relationships": [data]}

        if isinstance(data, str):
            logger.info("Detected top-level string => convert to single-element list of dict???")
            # 可能要尝试 json.loads ?
            # 具体看你需求

        # 其他情况保持原样
        return data

    @field_validator("relationships", mode="before")
    def validate_relationships(cls, v: Any) -> Optional[List[SchemaRelationship]]:
        v = cls.validate_array(v)
        if v is None:
            return None
        return [SchemaRelationship(**item) for item in v]

[yjing86]

open_llm_service.py generate_completion function uses response = self.client.beta.chat.completions.parse( model=self.settings.llm_model, messages=[{"role": "user", "content": prompt}], response_format=response_model, ) to validate LLM's response. However, this doesn't work and logger info is as below: WARNING:app.services.llm_service:LLM returned None response INFO: 127.0.0.1:50854 - "POST /api/v1/query HTTP/1.1" 200 OK ^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_base_client.py", line 1063, in _request return self._process_response( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_base_client.py", line 1162, in _process_response return api_response.parse() ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_response.py", line 319, in parse parsed = self._options.post_parser(parsed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/resources/beta/chat/completions.py", line 154, in parser return _parse_chat_completion( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 110, in parse_chat_completion "parsed": maybe_parse_content( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 161, in maybe_parse_content return _parse_content(response_format, message.content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 221, in _parse_content return cast(ResponseFormatT, model_parse_json(response_format, content)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_compat.py", line 169, in model_parse_json return model.model_validate_json(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/pydantic/main.py", line 656, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for StrArrayResponseModel Input should be an object [type=model_type, input_value=['quantum信息', '离散...', '计算机图形学'], input_type=list] For further information visit https://errors.pydantic.dev/2.10/v/model_type INFO:app.services.llm.openai_llm_service:generate_completion execution completed. INFO:app.services.llm_service:Raw response from LLM: None WARNING:app.services.llm_service:LLM returned None response INFO: 127.0.0.1:50854 - "POST /api/v1/query HTTP/1.1" 200 OK return api_response.parse() ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_response.py", line 319, in parse parsed = self._options.post_parser(parsed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/resources/beta/chat/completions.py", line 154, in parser return _parse_chat_completion( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 110, in parse_chat_completion "parsed": maybe_parse_content( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 161, in maybe_parse_content return _parse_content(response_format, message.content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/lib/_parsing/_completions.py", line 221, in _parse_content return cast(ResponseFormatT, model_parse_json(response_format, content)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/openai/_compat.py", line 169, in model_parse_json return model.model_validate_json(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/knowledge-table-main/lib/python3.11/site-packages/pydantic/main.py", line 656, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for StrArrayResponseModel Input should be an object [type=model_type, input_value=['quantum信息', '离散...', '计算机图形学'], input_type=list] For further information visit https://errors.pydantic.dev/2.10/v/model_type
I'm using Owen online model.

you must included " the response must be this json format : {'answer':['response']} " in the end of the query of the column.it's ok.

because the local llm response not include {'answer':.....}

I considered this and reviewed the code but found nothing unusual. However, modifying the llm_responses.py file proved to be helpful.

[lwing1966]

in ....../backend/src/app/services/llm/openai_prompts.py