Skip to content

typo in Readme #14

@Alice1069

Description

@Alice1069

what's the last 3 lines function in Readme?
"--swap-space
--max-model-len
--gpu-memory-utilization 0.99"

it seems duplicated with the above.

python /app/vllm/benchmarks/benchmark_throughput.py
--model /data/llm/Meta-Llama-3.1-405B-Instruct
--dtype float16
--gpu-memory-utilization 0.9
--num-prompts 2000
--distributed-executor-backend mp
--num-scheduler-steps 10
--tensor-parallel-size 8
--input-len 128
--output-len 128
--swap-space 16
--max-model-len 8192
--max-num-batched-tokens 65536
--swap-space
--max-model-len
--gpu-memory-utilization 0.99

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions