-
Notifications
You must be signed in to change notification settings - Fork 6
Description
我参考qwen3的论文,设置的参数,用vllm跑预测,参数设置如下:
llm = LLM(model=model_path,
tokenizer=model_path,
tensor_parallel_size=torch.cuda.device_count(),
max_model_len=32768,
dtype='bfloat16',
trust_remote_code=True,
seed=42,
gpu_memory_utilization=0.9)
sampling_params = SamplingParams( # qwen3 thinking
temperature=0.6,
top_p=0.95,
top_k=20,
max_tokens=32768,
# max_tokens=4096,
n=1,
stop=["<|im_end|>"])
prompt=messages = [
{"role": "user", "content": f"{question}\n{query_dic.get(lang)}"}
]
query_dic = {
"en": "Note: Please put the final answer in the
"zh": "注意:请将最终答案放在
"ar": "ملاحظة: يُرجى وضع الإجابة النهائية في
"bn": "বিঃদ্রঃ: অনুগ্রহ করে চূড়ান্ত উত্তরটি
"de": "Hinweis: Bitte setzen Sie die endgültige Antwort in
"es": "Nota: Por favor, coloque la respuesta final en el
"fr": "Remarque : Veuillez mettre la réponse finale dans le
"id": "Catatan: Silakan letakkan jawaban akhir di dalam
"it": "Nota: Per favore, metti la risposta finale nel
"ja": "注意:最終的な答えを
"ko": "참고: 최종 답안을
"ms": "Nota: Sila letakkan jawapan akhir dalam
"pt": "Nota: Por favor, coloque a resposta final no
"ru": "Примечание: Пожалуйста, поместите окончательный ответ в
"sw": "Kumbuka: Tafadhali weka jibu la mwisho katika
"te": "గమనిక: దయచేసి తుది జవాబును
"th": "หมายเหตุ: กรุณาใส่คำตอบสุดท้ายใน
"vi": "Lưu ý: Vui lòng đặt câu trả lời cuối cùng trong
}
请问这个有问题吗?en带权重的acc才跑到44%,leaderboard上是49.3%;望解答