-
Notifications
You must be signed in to change notification settings - Fork 70
Incompatibility error in compute capability 8.0 #21
Copy link
Copy link
Open
Description
https://github.com/MiniMax-AI/MiniMax-M2.5/blob/main/docs/sglang_deploy_guide.md
This document states that needs compute capability 7.0 or higher。
My gpu is an A100, and it prompts an incompatibility error during runtime: ValueError("type fp8e4nv not supported in this architecture. The supported fp8 dtypes are ('fp8e4b15', 'fp8e5')")
The cmd is:
python -m sglang.launch_server --model-path minimax/MiniMax-M2.5 --served-model-name minimax-m2.5 --port 10000 --host 0.0.0.0 --log-requests --log-requests-level 3 --log-requests-target ./sglang_logs/minimax --log-requests-format json --log-level=debug --crash-dump-folder ./sglang_crash_backup/minimax --tp-size 4 --mem-fraction-static 0.85 --context-length 196608 --reasoning-parser minimax-append-think --tool-call-parser minimax-m2 --trust-remote-codeHow can I solve this problem? Thank you
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels