How to run vllm baseline with benchmark_throughput.sh? #6
Description
What's the configuration to reproduce vllm baseline? I want to compare vllm baseline with DiffKV.
{{ message }}
What's the configuration to reproduce vllm baseline? I want to compare vllm baseline with DiffKV.