Skip to content

Incompatibility error in compute capability 8.0 #21

@tianqihou

Description

@tianqihou

https://github.com/MiniMax-AI/MiniMax-M2.5/blob/main/docs/sglang_deploy_guide.md
This document states that needs compute capability 7.0 or higher。

My gpu is an A100, and it prompts an incompatibility error during runtime: ValueError("type fp8e4nv not supported in this architecture. The supported fp8 dtypes are ('fp8e4b15', 'fp8e5')")

The cmd is:

python -m sglang.launch_server --model-path minimax/MiniMax-M2.5 --served-model-name minimax-m2.5 --port 10000 --host 0.0.0.0 --log-requests --log-requests-level 3 --log-requests-target ./sglang_logs/minimax --log-requests-format json --log-level=debug --crash-dump-folder ./sglang_crash_backup/minimax --tp-size 4 --mem-fraction-static 0.85 --context-length 196608 --reasoning-parser minimax-append-think --tool-call-parser minimax-m2 --trust-remote-code

How can I solve this problem? Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions