chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[amd] Add deterministic all-reduce kernel for AMD (ROCm) (#15340)

Browse Source 
Co-authored-by: Thomas Wang <1am9trash@gmail.com>
This commit is contained in:
sunxxuns
2025-12-18 23:36:03 -08:00
committed by GitHub
co-authored by Thomas Wang
parent 0e869f0868
commit f2d64e6782
12 changed files with 1466 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/environ.py
+5
View File
@@ -308,6 +308,11 @@ class Envs:
# Deterministic inference
SGLANG_ENABLE_DETERMINISTIC_INFERENCE = EnvBool(False)
# Use 1-stage all-reduce kernel on AMD (deterministic, fixed accumulation order)
# If not set: auto (enabled when --enable-deterministic-inference is on)
# Set to 1: force enable (even without --enable-deterministic-inference)
# Set to 0: force disable (use default Aiter AR even with --enable-deterministic-inference)
SGLANG_USE_1STAGE_ALLREDUCE = EnvBool(False)
SGLANG_FLASHINFER_PREFILL_SPLIT_TILE_SIZE = EnvInt(4096)
SGLANG_FLASHINFER_DECODE_SPLIT_TILE_SIZE = EnvInt(2048)
SGLANG_TRITON_PREFILL_TRUNCATION_ALIGN_SIZE = EnvInt(4096)