chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Deterministic Mode: Add 1-stage triton kernel for prefill (#11147)

Browse Source 
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Binyao Jiang <bijiang@linkedin.com>
This commit is contained in:
Stefan He
2025-10-20 01:47:36 +08:00
committed by GitHub
co-authored by Minglei Zhu Binyao Jiang
parent 7a020e0f3b
commit 4fff1ec1d9
5 changed files with 879 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/server_args.py
+2 -2
View File
@@ -1431,8 +1431,8 @@ class ServerArgs:
f"but you explicitly specified '{self.attention_backend}'."
)
# Currently, only FA3 supports radix cache. Support for other backends is in progress
if self.attention_backend != "fa3":
# Currently, only FA3 and Triton supports radix cache. Support for other backends is in progress
if self.attention_backend not in ["fa3", "triton"]:
self.disable_radix_cache = True
logger.warning(
f"Currently radix cache is not compatible with {self.attention_backend} attention backend for deterministic inference. It will be supported in the future."