[Bug] 在model配置中设置host_ip（ipv6的地址）、host_port，工具拉起报错 #185
Description
操作系统及版本
openEuler 22.03 (LTS-SP4)
安装工具的python环境
在anaconda/miniconda创建的python虚拟环境
python版本
其他
AISBench工具版本
AISBench执行命令
ais_bench --models vllm_api_stream_chat --datasets gsm8k_gen_4_shot_cot_chat_prompt --mode perf --num-prompts 1 --debug
模型配置文件或自定义配置文件内容
vi /xxx/benchmark/ais_bench/benchmark/configs/models/vllm_api/vllm_api_stream_chat.py
from ais_bench.benchmark.models import VLLMCustomAPIChat
from ais_bench.benchmark.utils.postprocess.model_postprocessors import extract_non_reasoning_content
models = [
dict(
attr="service",
type=VLLMCustomAPIChat,
abbr="vllm-api-stream-chat",
path="xxx/Qwen3-32B",
model="qwen-3-32b",
stream=True,
request_rate=0,
use_timestamp=False,
retry=2,
api_key="",
host_ip="::1",
host_port=8008,
#url="http://[::1]:8008",
max_out_len=512,
batch_size=1,
trust_remote_code=False,
generation_kwargs=dict(
temperature=0.01,
ignore_eos=False,
),
pred_postprocessor=dict(type=extract_non_reasoning_content),
)
]
预期行为
测评成功
实际行为
[2026-03-12 19:11:16,076] [ais_bench] [INFO] Loading gsm8k_gen_4_shot_cot_chat_prompt: /home/xxx/benchmark/ais_bench/benchmark/configs/./datasets/gsm8k/gsm8k_gen_4_shot_cot_chat_prompt.py
[2026-03-12 19:11:16,080] [ais_bench] [INFO] Loading example: /home/xxx/benchmark/ais_bench/benchmark/configs/./summarizers/example.py
[2026-03-12 19:11:16,123] [ais_bench] [INFO] Current exp folder: outputs/default/20260312_191107
[2026-03-12 19:11:16,123] [ais_bench] [INFO] Keeping the first 1 prompts for dataset [gsm8k]
[2026-03-12 19:11:16,138] [ais_bench] [INFO] Starting inference tasks...
[2026-03-12 19:11:16,139] [ais_bench] [INFO] Partitioned into 1 tasks.
[2026-03-12 19:11:16,139] [ais_bench] [INFO] Merging datasets with the same model and inferencer...
[2026-03-12 19:11:16,160] [ais_bench] [INFO] Launch TasksMonitor, PID: 481883, Refresh interval: 0.5, Run in background: True
[2026-03-12 19:11:26,021] [ais_bench] [INFO] Debug mode, print progress directly
[2026-03-12 19:11:26,023] [ais_bench] [INFO] Task [vllm-api-stream-chat/gsm8k]
[2026-03-12 19:11:26,510] [ais_bench] [INFO] Zero Retriever initialized, returning empty shot case for all queries
[2026-03-12 19:11:26,511] [ais_bench] [INFO] Apply ice template finished
[2026-03-12 19:11:26,513] [ais_bench] [INFO] Start warmup, run with concurrency: 1
Warmup: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1088.30case/s]
[2026-03-12 19:11:26,516] [ais_bench] [INFO] Warmup finished Total Count: 1 Success Count: 0 Failed Count: 1
Failed Reasons:
+------------------------------------------------------------------------------------------------------------+-------+
| Failed Reason | Count |
+------------------------------------------------------------------------------------------------------------+-------+
| After 2 retries, request failed with exception: InvalidUrlClientError: http://::1:8008/v1/chat/completions | 1 |
+------------------------------------------------------------------------------------------------------------+-------+
Traceback (most recent call last):
File "/home/xxx/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 704, in
raise e
File "/home/xxx/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 701, in
inferencer.run(task_state_manager)
File "/home/xxx/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 488, in run
self.warm_up(data_list, task_state_manager)
File "/home/xxx/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 467, in warm_up
raise AISBenchRuntimeError(
ais_bench.benchmark.utils.logging.exceptions.AISBenchRuntimeError: [TINFER-RUNTIME-001]warmup failed. Exit task because all warmup requests failed, failed reasons: {'After 2 retries, request failed with exception: InvalidUrlClientError: http://::1:8008/v1/chat/completions': 1} | Visit https://ais-bench-benchmark-rf.readthedocs.io/zh-cn/latest/faqs/error_codes.html#tinfer-runtime-001 for further help.
[2026-03-12 19:11:27,912] [ais_bench] [INFO] Inference tasks completed.
[2026-03-12 19:11:27,913] [ais_bench] [INFO] Summarizing performance results...
[2026-03-12 19:11:27,914] [ais_bench] [WARNING] Can't find details perf data of [vllm-api-stream-chat/gsm8k] in outputs/default/20260312_191107, use tmp cache data.
Traceback (most recent call last):
File "/root/miniconda3/envs/yjj_py312/bin/ais_bench", line 6, in
sys.exit(main())
^^^^^^
File "/home/xxx/benchmark/ais_bench/benchmark/cli/main.py", line 6, in main
task_manager.run()
File "/home/xxx/benchmark/ais_bench/benchmark/cli/task_manager.py", line 39, in run
workflow_executor.execute()
File "/home/xxx/benchmark/ais_bench/benchmark/cli/workers.py", line 433, in execute
worker.do_work(cfg)
File "/home/xxx/benchmark/ais_bench/benchmark/cli/workers.py", line 410, in do_work
summarizer.summarize()
File "/home/xxx/benchmark/ais_bench/benchmark/summarizers/default_perf.py", line 427, in summarize
details_perf_datas = self._load_details_perf_data(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/benchmark/ais_bench/benchmark/summarizers/default_perf.py", line 251, in _load_details_perf_data
raise FileMatchError(
ais_bench.benchmark.utils.logging.exceptions.FileMatchError: [SUMM-FILE-001]can't find detail perf data file. Can't find any details perf data file in work_dir, please check outputs/default/20260312_191107. | Visit https://ais-bench-benchmark-rf.readthedocs.io/zh-cn/latest/faqs/error_codes.html#summ-file-001 for further help.
(yjj_py312) [root@host169 benchmark]#
前置检查
- 我已读懂主页文档的快速入门，无法解决问题
- 我已检索过FAQ，无重复问题
- 我已搜索过现有Issue，无重复问题
- 我已更新到最新版本，问题仍存在