chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
10,161 Commits 1 Branch 0 Tags
e5edf222cdb9a99b51c9db44849441b64d480282
Commit Graph

296 Commits

Author SHA1 Message Date
Kangyan-Zhou
dc02e5bea7 [HiCache] Re-land spec v2 + decode KV cache offloading compatibility (#19615) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 13:58:31 -08:00
Kangyan-Zhou
dcf462cfba Revert "[HiCache] Enable spec v2 + decode KV cache offloading compatibility" (#19613) 2026-02-28 21:54:32 -08:00
Kangyan-Zhou
8167346609 [HiCache] Enable spec v2 + decode KV cache offloading compatibility (#19518) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-28 21:53:52 -08:00
Shangming Cai
366574b2b8 [PD] Cleanup BootstrapServer init and ready check (#19551) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-28 16:41:42 +08:00
Shangming Cai
b01f3590be [PD] Support PD with context parallel after refactor (#19504) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
 2026-02-28 13:11:15 +08:00
Liangsheng Yin
1a32c0db4d [Minor] Rename misleading chunked to reusing in ReqToTokenPool.alloc() (#19465) 2026-02-26 16:24:03 -08:00
Shangming Cai
288300aafd [PD] Tiny code cleanup for prefill info registering (#19414) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-27 01:35:26 +08:00
zhangheng
e4b708d3e9 [Spec V2] Support specV2 for mamba hybrid attention (#18808) 
Co-authored-by: Yi Zhong <207368749+vincentzed@users.noreply.github.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
 2026-02-27 00:36:01 +08:00
Shangming Cai
97f1fa5e6b [NPU] Fix disaggregation metadata buffer bootstrap_room_dtype for npu backend (#19423) 2026-02-26 21:10:50 +08:00
Shangming Cai
27fd014726 [PD] Add kv_cache_dtype consistency check for PD Disaggregation (#19407) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-26 17:15:58 +08:00
sglang-bot
6e82183f5a [Disagg] Route disagg prefill results through process_batch_result (#19364) 2026-02-25 18:38:39 -08:00
Liangsheng Yin
c60dcc40bb [Logging] Guard log_prefill_stats against idle batches in disagg prefill (#19361) 2026-02-25 13:31:52 -08:00
YAMY
08957c88ea [Logging] Fix prefill side logging in pd disagg (#19350) 2026-02-25 12:42:18 -08:00
YAMY
f75abb4521 [Fix][Qwen3.5] Fix KV cache slice transfer for GQA models with replicated KV heads (#19086) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-02-25 16:26:44 +08:00
Wang, Yi
3d879c69e9 refactor: extract device-to-backend mapping into get_default_distributed_backend (#19202) 
Signed-off-by: Wang, Yi <yi.a.wang@intel.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-02-24 23:50:26 -08:00
Liangsheng Yin
76d5410e01 [Disagg] Fix decode querying unregistered dp_rank when prefill dp_size is 1 (#19305) 
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
 2026-02-24 22:52:39 -08:00
Liangsheng Yin
ab0f608788 [PD-Disagg] Fix bootstrap server race condition when prefill workers not yet registered (#19288) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 20:22:16 -08:00
Liangsheng Yin
539f772f54 [PD-Disagg] Fully support external DP dispatch w/ PD-disaggregation mode. (#19268) 
Co-authored-by: Ratish P <114130421+ratish1@users.noreply.github.com>
 2026-02-24 19:58:01 -08:00
Shangming Cai
0fac2796b6 [PD-Disagg] Improve KVManager init across all backends (#19240) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-25 10:37:09 +08:00
Feng Su
3b89302277 Refactor: observability code cleanup (#17862) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2026-02-24 18:07:29 -08:00
siyu
245430eaac Encoder Global Cache Manager (#16137) 
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2026-02-24 18:05:43 -08:00
Liangsheng Yin
5e80027ac7 [PD-Disagg] Support FAKE transfer backend for more cases (#19215) 2026-02-23 20:50:43 -08:00
Liangsheng Yin
feb041f4e5 [PD-Disagg] Improve type hints across all conn.py (#19208) 2026-02-23 19:44:23 -08:00
Liangsheng Yin
ea7ef63e6d [PD-Disagg] Deduplicate common KVManager methods into CommonKVManager (#19205) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-02-23 18:34:21 -08:00
Liangsheng Yin
d160c5b9bb [PD-Disagg] Unify prefill info data transition flow, all with PrefillServerInfo (#19195) 2026-02-23 16:15:20 -08:00
Liangsheng Yin
2274bfebb1 [PD-Disagg] Support query dp rank from bootstrap server. (#19168) 
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
Co-authored-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
 2026-02-23 10:59:30 -08:00
Shangming Cai
eccee4c48e [PD] Change bootstrap_room metadata dtype from int64 to uint64 (#19141) 2026-02-22 14:20:16 +08:00
YAMY
cef353f338 [Fix] Quick fix for int32 overflow in Mooncakes' send_kvcache_slice (#19076) 2026-02-22 12:00:33 +08:00
Duyi-Wang
b0786cdf94 [AMD] Replace msgpack with msgspec in MORI-IO (#19007) 2026-02-19 23:04:15 -08:00
YAMY
8541b1118d [Fix][Qwen3.5] Pass max_mamba_cache_size to mamba pool in disaggregation decode path (#19002) 2026-02-20 14:31:26 +08:00
Neal Vaidya
ac0e493329 feat: add nsa and swa disagg support with nixl (#18939) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
 2026-02-18 13:13:26 +08:00
Ratish P
9a7d6be567 cleanup prefill metrics logging to fix dp-attn metrics (#18778) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-02-17 14:35:35 -08:00
Estrella-xx
1b3513a7e4 refactor FAKE transfer backend and remove --disaggregation-decode-enable-fake-auto parameter (#18345) 2026-02-16 17:27:02 +03:00
SoluMilken
07a24f1a38 update pre-commit config (#18860) 2026-02-16 00:18:31 +08:00
Liangsheng Yin
4474fb98b4 [PD-Disagg] Fix double free when prebuilt batch is aborted. (#18822) 2026-02-13 17:46:35 -08:00
Shangming Cai
1f39bf6523 [Bugfix] Add warnings when NSA indexer cache indice mismatch in PD module (#18727) 2026-02-13 15:20:05 +08:00
siyu
0b15f19927 [EPD] Add notification mechanism to fix server hang and add timeout env var (#18229) 2026-02-10 11:52:54 +08:00
Lianmin Zheng
b027c5aca6 [Auto Sync] Update cache_init_params.py (20260209) (#18502) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-09 14:49:41 -08:00
ishandhanani
01e3f4682e feat(kv-events): Add medium field to KV event types for storage tier tracking (#18205) 2026-02-09 12:39:15 -08:00
Zheng Wengang
68e31a3485 [BugFix][PD]Fix metadata_buffer_index leak when aborted in PD (#17483) 2026-02-09 11:34:29 +08:00
Shangming Cai
bffd765417 Refactoring Mooncake TE as a shared distributed component (#17810) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-09 10:53:11 +08:00
Neal Vaidya
f1ff697494 add hybrid model PD to NIXL connector (#16229) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-02-06 15:05:36 -08:00
ovidiusm
498d8d0680 NixlKVManager optimizations (#17654) 
Signed-off-by: Ovidiu Mara <ovidium@nvidia.com>
 2026-02-06 00:25:23 +08:00
Shangming Cai
afae4c7178 [PD] Minor code cleanup for mooncake backend (#18279) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-05 17:38:09 +08:00
Ch3ngY1
f730c18679 [PD] improve kv offset calculation for MHA model with different tp size (#18163) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-02-05 10:43:23 +08:00
Lewis
a45647bce1 [PD] feat: support mooncake intra-node nvlink kv transfer (#17866) 
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
 2026-02-03 17:47:52 +08:00
cctry
027f314050 [Fix] data race in req_to_token pool (#17850) 2026-02-02 14:38:15 -08:00
RangerCD
a480ca7ead fix: zmq_to_tokenizer encoder transfer when host listens to 0.0.0.0 (#17929) 2026-02-02 13:27:27 +08:00
siyu
f1824a957b [EPD][refactor]: introduce BaseMMReceiver for gRPC transport integration (#17921) 2026-02-02 11:37:32 +08:00
Simon (Jiyou) Li
8ed35df204 Add bootstrap_room validation to detect metadata corruption in PD disaggregation (#17430) 
Co-authored-by: 继优 <jiyou.ljy@alibaba-inc.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-02-02 10:43:50 +08:00