CUDA: Handle compute type for NVFP4 on cublass path (#29173)
* CUDA: Handle compute type for NVFP4 on cublass path Signed-off-by: ynankani <ynankani@nvidia.com> * Use BF16 compute type for quantized models if HW allows Signed-off-by: ynankani <ynankani@nvidia.com> * Set acc prec to bf16 for nvfp4 as it needs atleast bf16 range Signed-off-by: ynankani <ynankani@nvidia.com> * Update ggml/src/ggml-cuda/ggml-cuda.cu Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * preserve op_params for per-expert matmul Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com> Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
Y
ynankani committed
b56f34ab130b35c038ffa0acc81cb998a8b5f87c
Parent: c061df1
Committed by GitHub <noreply@github.com>
on 10/1/2026, 11:23:52 AM