KeyError: 'text' in the retrieve result #47
Description
Env. and Config.
I run the demo with the following instructions on my Ubuntu24.04:
conda env create --file conda_env.yaml
conda activate wikichat
python -m spacy download en_core_web_sm
inv demo --engine gpt-4o-mini
and the llm_config.yaml is as follow:
# WikiChat uses ChainLite (https://github.com/stanford-oval/chainlite), a wrapper around
# LangChain (https://github.com/langchain-ai/langchain) and LiteLLM (https://github.com/BerriAI/litellm) that enables easy and unified API access to hundreds of LLMs.
# This configuration file defines the setup for how ChainLite calls various LLM APIs, and how it logs LLM inputs/outputs.
# To configure it:
# Configure LLM endpoints under the `llm_endpoints` section, specifying the API base URL, version (if needed), API key (if needed),
# and the mapping of model names to their respective deployment identifiers. The name on the left-hand side of each mapping is "engine", the shorthand
# you can use in your code when calling llm_generation_chain(engine=...).
# The name on the right side-hand is the "model", the specific name that LiteLLM expects: https://docs.litellm.ai/docs/providers
# Note that "engine" names should be unique within this file, but "model" names do not have to be unique.
# Follow the examples provided for Azure, OpenAI, Groq, Together, Mistral, and local models as needed, and remove unused llm endpoints.
prompt_dirs: # relative to the location of this file
- "./" # Current directory
- "./pipelines/prompts"
litellm_set_verbose: false
prompt_logging:
log_file: "./data/prompt_logs.jsonl" # Path to the log file for prompt logs, relative to the location of this file
prompts_to_skip: # List of prompts to exclude from logging, relative to the location of this file
- "benchmark/prompts/user_with_passage.prompt"
- "benchmark/prompts/user_with_topic.prompt"
# Configuration for different LLM endpoints
llm_endpoints:
# Example of OpenAI models via Azure
# See more models at https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models.
# For Azure OpenAI, the value on the right hand side should be the "Deployment name" of your model
- api_base: https://[resource].openai.azure.com # Replace [resource] with your Azure OpenAI resource name
api_version: "2024-02-15-preview" # API version for Azure OpenAI
api_key: "AZURE_OPENAI_API_KEY" # This is the the name of the environment variable that contains your Azure OpenAI API key
engine_map:
gpt-35-turbo-instruct: azure_text/gpt-35-turbo-instruct
gpt-4o-mini: azure/gpt-4o-mini
# Example of OpenAI models via openai.com
# See more models at https://platform.openai.com/docs/models/. Note that OpenAI models don't need an "openai/" prefix in engine_map
- api_base: https://api.gptapi.us/v1
api_key: OPENAI_API_KEY # This is the the name of the environment variable that contains your OpenAI API key
engine_map:
gpt-35-turbo: gpt-3.5-turbo-0125
gpt-4o: gpt-4o
gpt-4o-mini: gpt-4o-mini
gpt-4: gpt-4-turbo-2024-04-09
# Example of fine-tuned OpenAI models via openai.com
# Visit https://platform.openai.com/finetune/ to learn more about fine-tuned OpenAI models
- api_base: https://api.gptapi.us/v1
api_key: OPENAI_API_KEY
prompt_format: distilled
engine_map:
gpt-35-turbo-finetuned: ft:gpt-3.5-turbo-1106:[model_id] # Replace [model_id] with your fine-tuned model id
# Example of open models via groq.com
# Groq has limited model availability, but a very fast inference on custom hardware
# See more models at https://console.groq.com/docs/models
- api_key: GROQ_API_KEY # This is the the name of the environment variable that contains your groq.com API key
engine_map:
llama-3-70b-instruct: groq/llama3-70b-8192
# Example of Claude models by Anthropic
# See more models at https://docs.anthropic.com/en/docs/about-claude/models
- api_base: https://api.anthropic.com/v1/messages
api_key: ANTHROPIC_API_KEY # This is the the name of the environment variable that contains your Anthropic API key
engine_map:
claude-sonnet-35: claude-3-5-sonnet-20240620
# Example of open models via together.ai
# See more models at https://docs.together.ai/docs/inference-models
# TODO non-instruct models don't work well because of LiteLLM's formatting issues, does not work with free accounts because of the 1 QPS limit
- api_key: TOGETHER_API_KEY # This is the the name of the environment variable that contains your together.ai API key
engine_map:
llama-2-70b: together_ai/togethercomputer/llama-2-70b
llama-3-70b-instruct: together_ai/meta-llama/Llama-3-70b-chat-hf
mixtral-8-7b: together_ai/mistralai/Mixtral-8x7B-v0.1
mixtral-8-7b-instruct: together_ai/mistralai/Mixtral-8x7B-Instruct-v0.1
mistral-7b: together_ai/mistralai/Mistral-7B-v0.1
# Example of models by Mistral
# See more models at https://docs.mistral.ai/platform/endpoints/
- api_base: https://api.mistral.ai/v1
api_key: MISTRAL_API_KEY
engine_map:
mistral-large: mistral/mistral-large-latest
mistral-medium: mistral/mistral-medium-latest
mistral-small: mistral/mistral-small-latest
mistral-7b-instruct: mistral/open-mistral-7b
mixtral-8-7b-instruct: mistral/open-mixtral-8x7b
# Local **distilled** models served via HuggingFace's text-generation-inference (https://github.com/huggingface/text-generation-inference/)
# This endpoint expects the model to have been fine-tuned with a specific data format generated from this repository.
# Use the next endpoint below instead if you are using general open models (with or without instructions and few-shot examples).
- api_base: http://127.0.0.1:5002
prompt_format: distilled
engine_map:
local_distilled: huggingface/local # The name after huggingface/ does not matter and is unused
# Local models served via HuggingFace's text-generation-inference (https://github.com/huggingface/text-generation-inference/)
# Use this endpoint if you want to host an open model like LLaMA, LLaMA-instruct, or open Mistral models etc.
- api_base: http://127.0.0.1:5002
engine_map:
local: huggingface/local # The name after huggingface/ does not matter and is unused
I only change the api_base of OpenAIModel, and add the API_KEYS file with the OPENAI_API_KEY
Problem
after i input the question, the demo will terminate with error, and it happens every time.
Traceback (most recent call last):
File "/home/splashcloud/workspace/rag_bench/WikiChat/command_line_chatbot.py", line 84, in <module>
asyncio.run(main(args))
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/splashcloud/workspace/rag_bench/WikiChat/command_line_chatbot.py", line 43, in main
new_agent_utterance, dialogue_state = await run_one_turn(
File "/home/splashcloud/workspace/rag_bench/WikiChat/pipelines/chatbot.py", line 42, in run_one_turn
dialogue_state = await chatbot.ainvoke(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 1160, in ainvoke
async for chunk in self.astream(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 976, in astream
_panic_or_proceed(done, inflight, step)
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 1193, in _panic_or_proceed
raise exc
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2536, in ainvoke
input = await step.ainvoke(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langgraph/utils.py", line 114, in ainvoke
ret = await self.afunc(input, **kwargs)
File "/home/splashcloud/workspace/rag_bench/WikiChat/pipelines/chatbot.py", line 398, in verify_stage
await verify_claim.ainvoke(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2536, in ainvoke
input = await step.ainvoke(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 4537, in ainvoke
return await self.bound.ainvoke(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3983, in ainvoke
return await self._acall_with_config(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1678, in _acall_with_config
output = await coro
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3930, in _ainvoke
output = await acall_func_with_variable_args(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3906, in f
return await run_in_executor(config, func, *args, **kwargs)
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 514, in run_in_executor
return await asyncio.get_running_loop().run_in_executor(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3900, in func
return call_func_with_variable_args(
File "/home/splashcloud/env/anaconda3/envs/wikichat/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 347, in call_func_with_variable_args
return func(input, **kwargs) # type: ignore[call-arg]
File "/home/splashcloud/workspace/rag_bench/WikiChat/pipelines/retriever.py", line 325, in batch_retrieve
result = RetrievalResult.retrieval_results_to_list(result)
File "/home/splashcloud/workspace/rag_bench/WikiChat/pipelines/retriever.py", line 68, in retrieval_results_to_list
results["text"],
KeyError: 'text'
I try to find the reason, and find the result send to retrieval_results_to_list function is as follow where my question is 'who is the mvp in 2023':
results:
{'results': [{'document_title': 'Center (basketball)', 'section_title': '2010s–present: Rise of stretch five and playmaking centers', 'content': "Another byproduct of the small ball revolution is the emergence of big men who possess guard-like skills. An example is Nikola Jokić, whose passing ability for a player his size made the Denver Nuggets contenders in the late 2010s and early 2020s, winning their first ever NBA Championship in 2023. Before 2021, the most recent center to win the NBA MVP award was Shaquille O'Neal in 2000. Since then the last four MVP winners have been centers, with Jokić winning three awards in 2021, 2022 and 2024, and Embiid in 2023.", 'last_edit_date': '2024-10-05', 'url': 'https://en.wikipedia.org/wiki/Center_(basketball)#2010s–present:_Rise_of_stretch_five_and_playmaking_centers', 'num_tokens': 0, 'block_metadata': {'language': 'en', 'type': 'text'}, 'similarity_score': 0.831, 'probability_score': 0.503, 'summary': []}, {'document_title': 'Joel Embiid', 'section_title': 'Palmarès', 'content': '2022, 2023\nNBA Most Valuable Player Award (in English: NBA Most Valuable Player Award): 1 National Basketball Association 2022-2023 (in English: 2022–23 NBA season)', 'last_edit_date': '2024-09-30', 'url': 'https://it.wikipedia.org/wiki/Joel_Embiid#Palmarès', 'num_tokens': 0, 'block_metadata': {'language': 'it', 'type': 'text'}, 'similarity_score': 0.82, 'probability_score': 0.497, 'summary': []}]}
I find that there is no 'text' key in it. So I want to know what went wrong.
Any help is greatly appreciated, please tell me if you need more information.