chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
10,885 Commits 1 Branch 0 Tags
7cc4e2dee8fe4d48d7c8c0f50b6237fecae6f230
Commit Graph

223 Commits

Author SHA1 Message Date
laoyao0822
7cc4e2dee8 docs(cp): design NSA prefill CP shared KV phase2 2026-04-24 06:06:31 +08:00
laoyao0822
e7a8c4b52f docs(cp): add NSA prefill CP dual all-gather explanation and Phase 1 plan 
- nsa_prefill_cp_all_gather.md: explains why there are two all-gathers
  in current NSA prefill CP (per-layer KV/index gather vs tail output
  hidden gather), with data flow diagrams and involved file lists
- nsa_prefill_cp_phase1_narrow_output_collection.md: detailed Phase 1
  design plan covering batch-level eligibility, fallback conditions,
  affected modules, KV transfer impact, risks, and test matrix
 2026-04-22 04:55:15 +08:00
laoyao0822
101100e25b feat(eplb): add eplb warmup-only mode 
Freeze expert layout after the initial EPLB warmup rebalances so heavy workloads avoid recurring rebalance memory spikes and OOMs.
 2026-04-08 06:46:26 +08:00
kpham-sgl
bc4aaab6a1 [Spec][Ngram] 2/N: Rename branch length to max trie depth (#21181) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 23:35:25 -07:00
kpham-sgl
6d160b42bb [Spec][Ngram] 1/N: Reference based Speculative Decoding refactor (#20393) 2026-03-22 00:55:10 -07:00
Xinyuan Tong
d1e95af282 Upgrade transformers==5.3.0 (#17784) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
Co-authored-by: Alison Shao <alisonshao@mac.lan>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-03-18 13:50:43 -07:00
ishandhanani
8f0f36c64b [1/2] Add ModelExpress coordination for remote instance weight loading - matching TP (#19920) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ishan Dhanani <ishan@dhanani.dev>
 2026-03-18 13:38:32 -07:00
Kangyan-Zhou
3d8fc9a0ca Revert "[Nvidia] Add trtllm mnnvl allreduce with unified flashinfer allreduce fusion api" (#20792) 2026-03-17 11:59:02 -07:00
Shu Wang
d35fea1b2b [Nvidia] Add trtllm mnnvl allreduce with unified flashinfer allreduce fusion api (#12787) 2026-03-17 10:02:45 -07:00
Teng Ma
7c498a6538 [DOC] add documents for encoder global mm cache (#20636) 2026-03-15 16:44:21 -07:00
Mook
23c191afb6 fix(docs): correct quantization documentation (#20301) (#20619) 2026-03-15 12:33:12 -04:00
Liangsheng Yin
fc7f9c1de7 Rename --stream-output to --incremental-streaming-output (#20614) 2026-03-14 23:22:33 -07:00
Matt Van Horn
d093e70067 [Doc] Add DSA/NSA attention backend to support matrix (#20326) 
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-11 13:40:35 -04:00
Yoray Zack
9991debde3 [Feature] Integrate Elastic NIXL-EP into SGLang (#19248) 
Signed-off-by: Barak Biber <bbiber@nvidia.com>
Signed-off-by: Yoray Zack <yorayz@nvidia.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Barak Biber <bbiber@nvidia.com>
 2026-03-11 17:37:43 +08:00
Liangsheng Yin
50953aea8d [Scheduler] Unify idle checks into is_fully_idle() and fix weight update test (#20296) 2026-03-10 17:50:23 -07:00
Ziang Li
76ee4bb98c [FlashInfer v0.6.4] [RL] Integrate FlashInfer mxfp8 gemm, MoE, and routed MoE (#19537) 2026-03-10 15:37:57 -07:00
shuwenn
5a11ae19c1 [CI] fix: notebook ci often OOM (#20199) 2026-03-09 22:32:41 -07:00
Brayden Zhong
591e61245a [Doc] Add smal table for GEMM backends (#20213) 2026-03-09 22:19:57 -07:00
YEJIN KIM
0fd9a57d80 [Doc] Verify and Modify some attention backend specs (#20210) 2026-03-09 23:05:52 +00:00
shuwenn
7bd3dd9270 fix: image URL in notebook to use raw.githubusercontent.com (#20100) 2026-03-07 13:28:20 -08:00
Bruce Changlong Xu
feda2b11c4 [AMD] Add AWQ AMD CI coverage and quantization platform compatibility docs (#19550) 2026-03-04 19:50:55 -08:00
Brayden Zhong
e2af840c3d Various SM120 improvements (#19721) 2026-03-03 16:46:13 -08:00
Sam (Kesen Li)
5b2e2750b5 Enable XQA for SM90 and SM120 (#17115) 
Co-authored-by: Xiaowei Wang <100599594+xiaoweiw-nv@users.noreply.github.com>
 2026-03-03 14:09:44 -08:00
zwang86
d6ac5f23cc [Docs] Add GDN attention backends matrix documentation (#19755) 
Co-authored-by: Zeyu Wang <zeyu.wang@yahooinc.com>
 2026-03-03 13:00:34 -08:00
Jasonzhang517
d939e26585 [model gateway][0/N] router EPD support: add encoder grpc server backend support (#16552) 
Co-authored-by: Zongyao Chen <ZongYao.Chen@linux.alibaba.com>
Co-authored-by: Zongyao Chen <solar1s@163.com>
 2026-03-03 19:38:15 +08:00
Yuwei An
0abb9f4176 Piecewise Cuda Graph Docs (#19738) 
Signed-off-by: yuweia <ayw.sirius19@gmail.com>
Co-authored-by: Wenyao Gao <wgao11@u.rochester.edu>
 2026-03-03 11:51:17 +08:00
shuwenn
bdffb027a8 [CI] fix: handle missing repo in lora notebook (#19700) 2026-03-02 10:27:32 -08:00
Shangming Cai
0a6678bf3a [PD] Remove unused server args for disaggregation (#19618) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-02 11:38:50 +08:00
shuwenn
e3e71f275a docs: refactor speculative decoding doc (#19186) 2026-03-01 22:03:20 -05:00
zwang86
f51ddba131 feat: add FA4 SM90 paged KV decode support & update attention docs (#18442) 
Co-authored-by: Zeyu Wang <zeyu.wang@yahooinc.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-03-02 09:12:19 +08:00
ympcMark
43fade5f69 [4/N] (Elastic EP) Back up Expert Weights in DRAM (#17374) 
Co-authored-by: UNIDY2002 <unidy2002@outlook.com>
 2026-02-27 15:59:13 +08:00
billishyahao
60eeef7370 [AMD][with CI Fix] support two batch overlapping for mori ep (#19216) 
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
Co-authored-by: kkHuang-amd <wunhuang@amd.com>
Co-authored-by: Feiyue Zhai <feiyue.zhai@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2026-02-25 02:14:08 -08:00
huangtingwei
d40cb2f725 [HiCache] Support heterogeneous tp for hicache storage (#18541) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-02-25 00:13:57 -08:00
Hubert Lu
17b0affbdf [AMD] Support --enable-aiter-allreduce-fusion on AMD GPUs (#13747) 
Co-authored-by: yctseng0211 <yctseng@amd.com>
 2026-02-24 23:11:55 -08:00
Liangsheng Yin
2274bfebb1 [PD-Disagg] Support query dp rank from bootstrap server. (#19168) 
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
Co-authored-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
 2026-02-23 10:59:30 -08:00
Baizhou Zhang
43f83525c0 Revert "[AMD] support two batch overlapping for mori ep #17953" (#19161) 2026-02-23 01:19:23 +08:00
billishyahao
fbb6098487 [AMD] support two batch overlapping for mori ep (#17953) 
Co-authored-by: kkHuang-amd <wunhuang@amd.com>
Co-authored-by: Feiyue Zhai <feiyue.zhai@amd.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2026-02-20 08:45:55 -08:00
Mohammad Miadh Angkad
2f592c3b18 [Doc] Add flashinfer_deepgemm to --fp8-gemm-backend (#18982) 2026-02-18 14:45:47 -05:00
Mengyang Liu
4f980f6f23 [Feature] Implement update_weights_from_disk for SGLang-D (Diffusion … (#18306) 
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
 2026-02-18 11:24:07 -08:00
Estrella-xx
1b3513a7e4 refactor FAKE transfer backend and remove --disaggregation-decode-enable-fake-auto parameter (#18345) 2026-02-16 17:27:02 +03:00
Rain Jiang
0ffd0a3995 Nsa trtllm mla sparse fp8 support with Deepseek v3.2 NVFP4 (#18389) 2026-02-16 09:29:54 +08:00
SoluMilken
07a24f1a38 update pre-commit config (#18860) 2026-02-16 00:18:31 +08:00
shuwenn
4cf4f0859f [Doc] Convert the speculative decoding notebook to markdow (#18395) 2026-02-14 18:18:56 -08:00
shuwenn
3299c4f9c1 [CI] feat: add early exit to wait_for_server when process dies (#18602) 2026-02-13 16:46:09 -08:00
dongjiyingdjy
8b4c364960 refactor context parallel state (#17213) 
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
 2026-02-13 23:18:17 +08:00
danielafrimi
e422bcaed8 [Mamba] Add float16 support for SSM cache dtype (#18444) 2026-02-12 11:27:47 +08:00
qianyue76
f06ab17a73 [diffusion] docs: consolidate diffusion documentation into docs (#18095) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: JiaxinD <djx2048@gmail.com>
 2026-02-11 16:55:07 -08:00
Baizhou Zhang
947927bdb5 [V3.2] Change default CP token split method to --round-robin-split (#18613) 2026-02-11 20:14:35 +08:00
赵晨阳
a2c38f7796 Enhance SMG guide with RL rollout systems benefits (#18588) 2026-02-10 20:20:45 -08:00
AlexZhao
3167bcc01c [Doc] Comprehensive Guide: Navigating DP, DPA, and SMG Best Practices (#18096) 
Co-authored-by: 赵海源 <zhaohaiyuan@xiaohongshu.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
 2026-02-10 18:31:28 -08:00