chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
10,885 Commits 1 Branch 0 Tags
7cc4e2dee8fe4d48d7c8c0f50b6237fecae6f230
Commit Graph

1178 Commits

Author SHA1 Message Date
laoyao0822
7cc4e2dee8 docs(cp): design NSA prefill CP shared KV phase2 2026-04-24 06:06:31 +08:00
laoyao0822
e7a8c4b52f docs(cp): add NSA prefill CP dual all-gather explanation and Phase 1 plan 
- nsa_prefill_cp_all_gather.md: explains why there are two all-gathers
  in current NSA prefill CP (per-layer KV/index gather vs tail output
  hidden gather), with data flow diagrams and involved file lists
- nsa_prefill_cp_phase1_narrow_output_collection.md: detailed Phase 1
  design plan covering batch-level eligibility, fallback conditions,
  affected modules, KV transfer impact, risks, and test matrix
 2026-04-22 04:55:15 +08:00
laoyao0822
101100e25b feat(eplb): add eplb warmup-only mode 
Freeze expert layout after the initial EPLB warmup rebalances so heavy workloads avoid recurring rebalance memory spikes and OOMs.
 2026-04-08 06:46:26 +08:00
kpham-sgl
bc4aaab6a1 [Spec][Ngram] 2/N: Rename branch length to max trie depth (#21181) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 23:35:25 -07:00
Xiaoyu Zhang
766d225fcc Add SGLang CUDA crash API logging inspired by FlashInfer (#20910) 2026-03-22 16:39:40 +08:00
kpham-sgl
6d160b42bb [Spec][Ngram] 1/N: Reference based Speculative Decoding refactor (#20393) 2026-03-22 00:55:10 -07:00
Rabinovich
3798a8c88d docs: add out-of-tree model integration guide (#21050) 
Co-authored-by: Yixiao Zeng <yixiao.zeng@xiaopeng.com>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
 2026-03-20 20:07:46 -07:00
Lianmin Zheng
2d7a262ca3 ci: rename 1/2-gpu-runner labels to 1/2-gpu-h100 (#21008) 2026-03-20 06:04:15 -07:00
Cao E
274581fb77 Add support for more batch sizes in cpu_graph_runner (#13881) 2026-03-19 09:50:56 -07:00
blzheng
cbea9f6909 [CPU] improve numa memory binding (#19666) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-18 22:15:50 -07:00
Xinyuan Tong
d1e95af282 Upgrade transformers==5.3.0 (#17784) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
Co-authored-by: Alison Shao <alisonshao@mac.lan>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-03-18 13:50:43 -07:00
ishandhanani
8f0f36c64b [1/2] Add ModelExpress coordination for remote instance weight loading - matching TP (#19920) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ishan Dhanani <ishan@dhanani.dev>
 2026-03-18 13:38:32 -07:00
Lianmin Zheng
46a392658e Refine RL & Post-Training description in README (#20877) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-18 12:43:42 -07:00
Xiaoyu Zhang
20a23e3173 [SKILL] Refine kernel authoring docs and validate add-jit-kernel / add-sgl-kernel end to end with Codex (#20867) 2026-03-18 23:00:33 +08:00
Ke Bao
c42da50289 Update test guide to contribution guide (#20805) 2026-03-18 13:25:16 +08:00
Qiaolin Yu
c5d2528bff Revert "[AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars." (#20797) 2026-03-17 17:28:09 -07:00
Kangyan-Zhou
3d8fc9a0ca Revert "[Nvidia] Add trtllm mnnvl allreduce with unified flashinfer allreduce fusion api" (#20792) 2026-03-17 11:59:02 -07:00
Shu Wang
d35fea1b2b [Nvidia] Add trtllm mnnvl allreduce with unified flashinfer allreduce fusion api (#12787) 2026-03-17 10:02:45 -07:00
Duyi-Wang
385a35bd11 [AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars. (#20647) 2026-03-17 01:13:42 -07:00
DefTruth
025691cd9e [diffusion] chore: bump up cache-dit & support quant for diffusers backend (#20361) 2026-03-17 12:51:31 +08:00
amote-i
210d0fbaef Update ascend docs (#20674) 2026-03-16 20:14:26 -07:00
Xiaoyu Zhang
15097c5c3b Release sglang kernel 0.4.0 (#20440) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-03-16 20:34:58 +08:00
amote-i
da1793f63a update ascend feature docs (#20506) 2026-03-15 20:09:20 -07:00
AnonTokyo
e9fae69e5f docs: align environment variable reference with environ defaults (#20419) 2026-03-15 18:07:29 -07:00
Teng Ma
7c498a6538 [DOC] add documents for encoder global mm cache (#20636) 2026-03-15 16:44:21 -07:00
Mook
23c191afb6 fix(docs): correct quantization documentation (#20301) (#20619) 2026-03-15 12:33:12 -04:00
Liangsheng Yin
fc7f9c1de7 Rename --stream-output to --incremental-streaming-output (#20614) 2026-03-14 23:22:33 -07:00
Matt Van Horn
6c5bf53a36 [Doc] Clarify that --chat-template is required for Qwen3-Reranker (#20596) 
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-14 23:43:48 +00:00
Baizhou Zhang
39008955ff Revert "[AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars." (#20602) 2026-03-14 12:12:42 -07:00
Duyi-Wang
0eea80bc00 [AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars. (#20453) 2026-03-13 14:03:17 -07:00
StonyPort
d4e68ead1d [quant] Ignore FP8 quantization layers (#20340) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-13 13:59:39 +08:00
Mook
2720ea2667 [Typo] Fix H200 doc links pointing to H20 section in deepseek_v3.md (#20383) 2026-03-11 13:35:20 -07:00
Matt Van Horn
d093e70067 [Doc] Add DSA/NSA attention backend to support matrix (#20326) 
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-11 13:40:35 -04:00
R0CKSTAR
dae5c6cadf [diffusion] doc: add Moore Threads as a supported vendor (#20146) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-03-11 10:15:15 -07:00
Thomas
05e40922b3 [Doc] Fix wrong link and cmd description (#20365) 2026-03-11 11:30:45 -04:00
Артем Савкин
ed42af99a9 [NPU] [Quantization] w4a4 MoE layer support (#18924) 2026-03-11 16:52:35 +03:00
Yoray Zack
9991debde3 [Feature] Integrate Elastic NIXL-EP into SGLang (#19248) 
Signed-off-by: Barak Biber <bbiber@nvidia.com>
Signed-off-by: Yoray Zack <yorayz@nvidia.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Barak Biber <bbiber@nvidia.com>
 2026-03-11 17:37:43 +08:00
Xuhao Zhang
57b093dc34 [NPU]MindSpore backend support eagle3 (#17098) 
Co-authored-by: wangtiance <tiancew@qq.com>
Co-authored-by: Tiance Wang <wangtiance@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-03-11 09:11:19 +03:00
Aleksi Vesanto
c8bbe5010a [diffusion] feat: add AITER Sage attention backend (#20178) 2026-03-11 12:17:45 +08:00
Polisetty V R K Jyothendra Varma
b2dd104ade [Intel GPU] Upgrade pytorch xpu version to 2.10 (#20254) 
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
 2026-03-10 18:47:25 -07:00
Liangsheng Yin
50953aea8d [Scheduler] Unify idle checks into is_fully_idle() and fix weight update test (#20296) 2026-03-10 17:50:23 -07:00
Ziang Li
76ee4bb98c [FlashInfer v0.6.4] [RL] Integrate FlashInfer mxfp8 gemm, MoE, and routed MoE (#19537) 2026-03-10 15:37:57 -07:00
R0CKSTAR
db97f193b7 [diffusion][llm] macOS support (#19549) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-03-10 13:11:07 -07:00
shuwenn
5a11ae19c1 [CI] fix: notebook ci often OOM (#20199) 2026-03-09 22:32:41 -07:00
Brayden Zhong
591e61245a [Doc] Add smal table for GEMM backends (#20213) 2026-03-09 22:19:57 -07:00
YEJIN KIM
0fd9a57d80 [Doc] Verify and Modify some attention backend specs (#20210) 2026-03-09 23:05:52 +00:00
Baizhou Zhang
be63f982b7 [V32/GLM5] Control the threshold of applying dense attention with an environ (#20062) 2026-03-09 14:36:10 -07:00
AMD-yanfeiwang
f0153ad225 [AMD][Feature] support fp4 dispatch and fp8 combine in moriep (#19757) 
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
 2026-03-09 12:52:05 -07:00
Yuhao Yang
57f28fda90 [diffusion] chore: add diffusion new model skill (#19605) 2026-03-09 09:45:23 +08:00
Junhao Liu
7662b8b919 [diffusion] feat: implement upscaling (#19723) 2026-03-09 02:06:40 +08:00