chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Bug-fix] Fix gpu fault when run the test with dp-attention-enabled and max-concurrency is over 256 (#20399)

Browse Source 
Co-authored-by: wunhuang <wunhuang@amd.com>
This commit is contained in:
kk
2026-03-12 17:32:03 +08:00
committed by GitHub
parent 4e5ca92249
commit 318a40fdfb
1 changed files with 5 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
python/sglang/srt/layers/attention/aiter_backend.py
View File

@@ -217,7 +217,11 @@ class AiterAttnBackend(AttentionBackend):
)
global _use_mla_ps_kernel, fast_mode, intra_batch_mode
if self.num_head == 32:
# current mla_decode_fwd onln support fake-nps in self.num_head == 16
# so all num_head size does not use qh16 kernel to simulate
# it should not use fake-nps (fast_mode = False, intra_batch_mode = True)
# it will cause gpu-fault or accuracy issue
if self.num_head == 32 or self.num_head == 128:
fast_mode = True
intra_batch_mode = False