GroupedGemm: NVFP4 via cuBLAS

[ptrendx]

Implement GroupedGemm for NVFP4 format using cuBLAS kernels. Ensure integration with GroupedTensor utilities and grouped quantization pathways for Sync-Free MoE training.

[CodersAcademy006]

Hi @ptrendx,

My rough understanding is that this would involve extending the grouped GEMM dispatch to recognize NVFP4 tensors coming from the grouped quantization pipeline and routing them to the appropriate cuBLAS/cuBLASLt kernels, while keeping it compatible with the existing GroupedTensor utilities.

Does the main missing piece live in the grouped GEMM dispatch layer itself, or in the NVFP4 tensor/quantization integration?

Happy to dig into this area and help prototype the integration if that would be useful. 🙂