[Docs] Add GDN attention backends matrix documentation (#19755)

Co-authored-by: Zeyu Wang <zeyu.wang@yahooinc.com>
This commit is contained in:
zwang86
2026-03-03 13:00:34 -08:00
committed by GitHub
co-authored by Zeyu Wang
parent cedb86a950
commit d6ac5f23cc
@@ -77,6 +77,28 @@ MLA page-size constraints:
- Cutlass MLA: page_size = 128.
- TRTLLM MLA: page_size ∈ {32, 64}.
### GDN Attention Backends
GDN (Gated Delta Network) is a linear attention mechanism with O(n) complexity, used in hybrid models that alternate GDN linear attention layers with standard full attention layers. GDN is **not** selected via `--attention-backend`; it is automatically activated when the model architecture requires it (e.g., Qwen 3.5, Qwen 3 Next, Jet Nemotron, Jet VLM).
The GDN linear attention layers have their own kernel backends, selected via `--linear-attn-backend` (default: `triton`). You can override the kernel per phase with `--linear-attn-decode-backend` and `--linear-attn-prefill-backend`.
| **Backend** | **Decode** | **Prefill / Extend** | **Spec Decoding (Target Verify)** |
|--------------------------|------------|----------------------|-----------------------------------|
| **Triton (CUDA)** | ✅ | ✅ | ✅ |
| **Triton (AMD/ROCm)** | ✅ | ✅ | ✅ |
| **Triton (NPU)** | ✅ | ✅ | ❌ |
| **Triton (CPU)** | ✅ | ✅ | ❌ |
| **CuTe DSL (CUDA only)**| ✅ | ❌ | ❌ |
```{important}
GDN models are hybrid: the full-attention layers still require a standard `--attention-backend`. Platform constraints for the full-attention backend on hybrid GDN models:
- **Blackwell (e.g., B200)**: `triton`, `trtllm_mha`, or `fa4` only.
- **NPU (Ascend)**: `ascend` only.
- **AMD (ROCm)**: `triton` recommended.
- **Other CUDA (Hopper, Ampere, etc.)**: auto-selection works; no special constraints.
```
### Hybrid attention (different backends for prefill vs decode) (Experimental)
```{warning}
@@ -278,6 +300,10 @@ To add a new attention backend, you can learn from the existing backends
(`python/sglang/srt/layers/attention/triton_backend.py`, `python/sglang/srt/layers/attention/flashattention_backend.py`)
and follow the steps below.
```{note}
Linear attention kernel backends (GDN, KDA) follow a different pattern. They implement `LinearAttnKernelBase` in `python/sglang/srt/layers/attention/linear/kernels/` and are dispatched by `GDNKernelDispatcher` / `KDAKernelDispatcher` rather than registered via `@register_attention_backend`.
```
1. Run without cuda graph. Support the two forward functions
- forward_extend
- Will be used for prefill, prefill with KV cache, and target verification