chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836)

Browse Source
This commit is contained in:
fzyzcjy
2025-04-18 12:38:26 +08:00
committed by GitHub
parent 1effba4c70
commit 53dcf38876
4 changed files with 31 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/model_executor/model_runner.py
View File

@@ -159,6 +159,7 @@ class ModelRunner:
"speculative_accept_threshold_acc": server_args.speculative_accept_threshold_acc,
"disable_radix_cache": server_args.disable_radix_cache,
"flashinfer_mla_disable_ragged": server_args.flashinfer_mla_disable_ragged,
"moe_dense_tp_size": server_args.moe_dense_tp_size,
"debug_tensor_dump_output_folder": server_args.debug_tensor_dump_output_folder,
"debug_tensor_dump_inject": server_args.debug_tensor_dump_inject,
"n_share_experts_fusion": server_args.n_share_experts_fusion,