[train_rlhf_llama] When using vllm，Time-consuming optimization is not obvious as expectation.

[tkqie]

When running the train_rlhf_llama.sh script for LLaMA2-7B, the performance gain from using vLLM as the inference backend is not significant compared to Megatron (approximately a 20% improvement). According to some other reports, the expected improvement should be around 200%. How can it be promoted?

[adoda]

Our tests show an improvement of about 2 times. You can compare this with the configuration and performance data provided in our documentation to see if there are any differences.