[ROCm] Replace compile-time warp size with runtime query in host code

[sstamenk]

Summary

• Replace the compile-time BNB_WARP_SIZE macro with a runtime bnb_host_warp_size() query in all host-side dispatch code. The macro is only correctly defined during device-code compilation passes and was silently wrong in host code on ROCm 7.x (which removed __AMDGCN_WAVEFRONT_SIZE). On RDNA which utilizes a warp size of 32 this logic was defaulting to 64.
• Fix the blocksize=64 4-bit quantization dispatch on HIP to select the correct kernel based on the actual device warp size, instead of unconditionally using kQuantizeBlockwiseSmall with a hardcoded 64-thread launch.
• Improve the BNB_WARP_SIZE fallback chain in common.cuh for ROCm 7.0+ where __AMDGCN_WAVEFRONT_SIZE is no longer emitted: fall back to __GFX9__ (CDNA = 64), then default to 32 (RDNA).

Problem

On RDNA GPUs (gfx10xx/gfx11xx/gfx12xx, wavefront size 32), the old default of BNB_WARP_SIZE = 64 caused two classes of failures:

1. blocksize=32 crash (Fatal Python error: Aborted): kQuantizeBlockwiseSmall was compiled with THREADS=64 but launched with only 32 threads, causing CUB cooperative operations to abort.
2. blocksize=64 garbage output (mean error ~0.73 vs threshold 0.11): After fixing the macro default to 32, kQuantizeBlockwiseSmall now compiled with THREADS=32 but was still launched with 64 threads (hardcoded in the blocksize=64 HIP dispatch path), producing corrupted quantization output. Unit tests test_4bit_compressed_stats and test_4bit_quant with blocksize=64 were broken on RDNA as a result.

The root issue is that BNB_WARP_SIZE is architecture-specific and only valid inside device code, but was being used in host-side kernel launch configuration.

Changes

csrc/common.cuh — Improved warp size fallback for device code:

• __AMDGCN_WAVEFRONT_SIZE (preferred, but removed in ROCm 7.0)
• __GFX9__ → 64 (CDNA)
• Default → 32 (RDNA and others)

csrc/ops.cu — Runtime warp size for host code:

• Added bnb_host_warp_size(): queries device 0 with hipDeviceAttributeWarpSize. Returns 32 on CUDA.
• gemm_4bit_inference_naive: replaced BNB_WARP_SIZE == 64 with bnb_host_warp_size() == 64.
• quantizeBlockwise for blocksize=64 on HIP: dispatch to kQuantizeBlockwiseSmall only when runtime warp size is 64 (CDNA); fall through to kQuantizeBlockwise<64, 2> otherwise (RDNA, same as CUDA path).

Test plan

• Verify all relevant unit tests pass on RDNA
• Confirm no unit test regression on gfx9xx

[github-actions]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[matthewdouglas]

Thanks! Looks good apart from minor lint issue. @Abdennacer-Badaoui do you have any feedback?

[sstamenk]

@matthewdouglas I have a follow up to this at #1887
Would highly appreciate feedback from @Abdennacer-Badaoui there as well considering he originally implemented the kernel.

[Abdennacer-Badaoui]

Looks good to me. Thanks @sstamenk !
I will look at your follow up PR now.

[Abdennacer-Badaoui]

LGTM! The separation of compile-time BNB_WARP_SIZE (for device code) and runtime bnb_host_warp_size() (for host-side launch config) is the right approach for correct RDNA/CDNA dispatch.
Minor nit: the static warp_size cache has a benign data race (could use std::atomic) but not a blocker.

[Abdennacer-Badaoui]

can you fix the linting @sstamenk , thanks!

[sstamenk]

Good catch, I've changed it to utilize atomics.
The worst case I can see is that we do the hip query more than once which shouldn't really cause problems since we always query device 0 but I'm not opposed to guarding it with atomic reads/writes.