chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Reduce the overhead of nccl symmetric memory (#12524)

Browse Source 
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
This commit is contained in:
Lianmin Zheng
2025-11-03 11:56:27 -08:00
committed by GitHub
parent d36639eec7
commit 7a21d8b276
14 changed files with 219 additions and 154 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/sglang/srt/server_args.py
View File

@@ -3174,7 +3174,7 @@ class ServerArgs:
parser.add_argument(
"--enable-torch-symm-mem",
action="store_true",
help="Enable using torch symm mem for all-reduce kernel and fall back to NCCL. Only supports CUDA device SM90 and above. SM90 supports world size 4, 6, 8. SM10 supports world size 6, 8.",
help="Enable using torch symm mem for all-reduce kernel and fall back to NCCL. Only supports CUDA device SM90 and above. SM90 supports world size 4, 6, 8. SM100 supports world size 6, 8.",
)
parser.add_argument(
"--disable-overlap-schedule",