[ROCm][FP4 BMM] integrate FP4 BMM

[zejunchen-zejun]

Integrate the FP4 BMM and unify the env flag VLLM_ROCM_USE_AITER_BMM.
When VLLM_ROCM_USE_AITER_BMM=1(default), the logic is:
When weight in attention part is BF16 dtype, the FP8 BMM is called.
When weight in attention part is U8 dtype, the FP4 BMM is called.

When VLLM_ROCM_USE_AITER_BMM=0, the torch BMM is used.

For model DeepSeek-R1-MXFP4-Preview, whose kv_b_proj weight is U8, the FP4 BMM is used. Here is the associated performance and accuracy.

The FP4 accuracy is:

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.9515	±	0.0059
		strict-match	5	exact_match	↑	0.9507	±	0.0060

The FP4 BMM performance is:
Request throughput (req/s): 1.22
Mean TTFT (ms): 6474.98
Mean TPOT (ms): 44.95

The baseline performance(FP8 BMM) is:
Request throughput (req/s): 1.21
Mean TTFT (ms): 6732.17
Mean TPOT (ms): 45.16

[zejunchen-zejun]

Hi, @ZhiweiYan-96
Here is the PR for the FP4 BMM.
The dequant method in post processing is not very efficient.
You can modify it to use switch bit and subbyte storage for post processing U8 weight.

Finally you can upstream this PR to vllm community.

[ZhiweiYan-96]

there should be some utils in vllm for quant&dequant

[github-actions]

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!