chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[RL] support weight reload for low-bit rollout (#9650)

Browse Source 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
This commit is contained in:
Peng Zhang
2025-12-10 15:44:01 +08:00
committed by GitHub
parent b0a25d0913
commit 21028b5507
7 changed files with 581 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/model_executor/model_runner.py
View File

@@ -764,6 +764,7 @@ class ModelRunner:
remote_instance_weight_loader_seed_instance_service_port=self.server_args.remote_instance_weight_loader_seed_instance_service_port,
remote_instance_weight_loader_send_weights_group_ports=self.server_args.remote_instance_weight_loader_send_weights_group_ports,
modelopt_config=modelopt_config,
rl_quant_profile=self.server_args.rl_quant_profile,
)
if self.device == "cpu":
self.model_config = adjust_config_with_unaligned_cpu_tp(