chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[V32/GLM5] Control the threshold of applying dense attention with an environ (#20062)

Browse Source
This commit is contained in:
Baizhou Zhang
2026-03-09 14:36:10 -07:00
committed by GitHub
parent d39ed074cf
commit be63f982b7
6 changed files with 32 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/references/environment_variables.md
View File

@@ -91,6 +91,8 @@ SGLang supports various environment variables that can be used to configure its
| --- | --- | --- |
| `SGLANG_NSA_FUSE_TOPK` | Fuse the operation of picking topk logits and picking topk indices from page table | `true` |
| `SGLANG_NSA_ENABLE_MTP_PRECOMPUTE_METADATA` | Precompute metadata that can be shared among different draft steps when MTP is enabled | `true` |
| `SGLANG_USE_FUSED_METADATA_COPY` | Control whether to use fused metadata copy kernel for cuda graph replay | `true` |
| `SGLANG_NSA_PREFILL_DENSE_ATTN_KV_LEN_THRESHOLD` | When the maximum kv len in current prefill batch exceeds this value, the sparse mla kernel will be applied, else it falls back to dense MHA implementation. Default to the index topk of model (2048 for DeepSeek V3.2) | `2048` |
## Memory Management