chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
8,997 Commits 1 Branch 0 Tags
ad1b4e4728236269383850ec948124e64fd76926
Commit Graph

4846 Commits

Author SHA1 Message Date
Glen Liu
ad1b4e4728 [Feature] overlap LoRA weight loading with compute (#15512) 2026-01-19 10:43:17 +08:00
Koushik Dutta
d3eafc7357 [GLM 4.7] Add RTX 6000 Pro aka sm120 (#17235) 
Co-authored-by: root <root@ubuntu-nvidia.localdomain>
 2026-01-18 13:33:19 -08:00
Jinyan Chen
e00b43442d [jit-kernel] Add CuTe DSL GDN Decode Kernel (#15631) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2026-01-18 12:54:36 -08:00
Todobe
733de6be31 [NPU]Support GPT-OSS for NPU (#14197) 2026-01-19 04:13:41 +08:00
Jerry Ji
93433726eb [Refactor] Split out deepseek v2 weight loader function into mixin (#16649) 2026-01-19 00:41:59 +08:00
Ch3ngY1
bb6055b43c [Bugfix] Fix PD accuracy when MTP is not configured on the prefill node (#17212) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-01-18 23:37:58 +08:00
b8zhong
4df74eb576 [Refactor] Add -fp4-gemm-backend to replace SGLANG_FLASHINFER_FP4_GEMM_BACKEND (#16534) 
Co-authored-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
 2026-01-18 23:25:46 +08:00
Xinyuan Tong
2069050d3f fix: Handle multiple named chat templates in HuggingFace tokenizers (#17236) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-18 17:20:04 +08:00
YAMY
a45e0e5df4 [SPEC_V2] Enable cudagraph draft_extend for trtllm_mla_backend and Acclen Fix for DP under cudagraph mode (#16974) 2026-01-18 15:56:21 +08:00
Yuan Luo
6d29d8ab16 [VLM][Reland] Refactor load_mm_data to improve performance (#16152) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-18 14:11:17 +08:00
Ke Bao
e499258e97 Use swa radix cache and memory pool for gpt-oss model (#17261) 2026-01-18 13:47:07 +08:00
Zehuan Li
d2c863878c [DLLM] Implement initial dynamic batching for diffusion LLM (#14883) 2026-01-17 16:48:15 +08:00
Yi Zhang
737a1183d6 [BUGFIX] fix radix cache memory consumption to avoid OOM (#17191) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-01-17 16:47:37 +08:00
Nan Jiang
dd99f818e0 fix: fix regression and unclear pattern (#16561) 2026-01-16 23:21:42 -08:00
siyu
e18e0057a4 Add an env var to allow transferring small metadata via TCP for PD (#16951) 2026-01-16 18:32:22 -08:00
Stefan He
43779f27b7 Disable PCG for draft worker (#16354) 2026-01-17 09:56:56 +08:00
Baizhou Zhang
8b9e9357fe [2/n] deepseek_v2.py Refactor: Migrate MHA forward method in deepseek_v2.py (#16817) 2026-01-17 09:36:25 +08:00
b8zhong
d36f6f043c [Fix] flashinfer_trtllm intermediate_size assertion with Qwen3 + TP=8 (#16824) 2026-01-17 09:24:05 +08:00
Baizhou Zhang
a04675892e Update flashinfer to 0.6.1 (#15551) 2026-01-17 00:48:30 +08:00
Yongfei Xu
82a1b645ba [DeepSeek V3.1/V3.2] Optimize fused moe configs for H20 & H20-3E based on swapab (#17133) 2026-01-17 00:10:52 +08:00
billishyahao
6f10e17b4a [bugfix] fix qwen3-next alt_stream none issue (#17016) 2026-01-16 22:40:25 +08:00
Nicolas Castet
9d8bbd4223 Add clear error message when OOM with symmetric memory (#17038) 2026-01-16 20:44:37 +08:00
Raghav Ravishankar
daea51385d Add AFMoE model implementation (#13216) 2026-01-16 20:35:42 +08:00
StonyPort
3355b6e21b feat: add request queued timeout (#17143) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-01-16 17:55:09 +08:00
Zhiqiang Xie
d9ed80b9f1 fix AMD CI failure of NUMA binding (#17184) 2026-01-16 00:21:04 -08:00
YAMY
daa4841e86 [ConfigArgumentMerger] Improve ConfigArgumentMerger compatibility with external callers (#17051) 2026-01-16 15:32:42 +08:00
Hudson Xing
21ee597e4a ci: enable offline mode when local cache is complete to avoid HF Hub … (#16121) 2026-01-15 20:15:33 -08:00
shuwenn
8ec160ed46 feature: support uvicorn access log filter(disable logging /metrics) (#15513) 2026-01-15 20:00:06 -08:00
Lianmin Zheng
e7dc85c50b Fix grammar sync across TP ranks (#17100) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-01-15 18:38:01 -08:00
PiteXChen
8b22deef5b fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 15:14:38 -08:00
JinYan Su
72e2f70ef7 feat(hicache): support numa detect to reduce long tail latency (#11028) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 14:11:49 -08:00
huangtingwei
77fc4c4a53 Add mooncake store read/write bandwidth logs (#10598) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Lei Huang <lei.huang@novita.ai>
 2026-01-15 12:15:41 -08:00
sglang-bot
000ad42225 chore: bump sgl-kernel version to 0.3.21 (#17075) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-01-15 12:41:17 +08:00
Hanming Lu
9d5f16d456 [SWA] fix swa radix cache match_len_since_tombstone update when hits swa_tombstone (#17061) 2026-01-15 11:28:33 +08:00
Ke Bao
7f8a58fffb Refactor prefix cache type checking (#17028) 2026-01-15 11:28:13 +08:00
b8zhong
4346db5faf [Fix] Remove assertion for padding for NVFP4 weight scales to fix GLM 4.5 NVFP4 (#12497) 2026-01-15 08:57:14 +08:00
Артем Савкин
424a380077 [NPU] NPU quantization refactoring & more quantization formats support (#14504) 
Co-authored-by: TamirBaydasov <mr.jeijy@gmail.com>
Co-authored-by: Tamir Baydasov <41994229+TamirBaydasov@users.noreply.github.com>
Co-authored-by: Савкин Артем <savkinartem@MacBook-Air-Viktoria.local>
Co-authored-by: Edward Shogulin <edward.shogulin@gmail.com>
 2026-01-15 04:25:15 +08:00
Simo Lin
f091858304 [gRPC] Add GetLoads RPC for comprehensive load metrics (#17087) 2026-01-14 12:05:10 -08:00
Aurick Qiao
5b1215d9da fix session request with None tokenizer (#16278) 2026-01-14 10:41:08 -08:00
Simo Lin
b3a3f51320 [API] Add /v1/loads endpoint for load metrics (#16976) 2026-01-14 09:13:20 -08:00
shuwenn
de94d793ad feat: support qwen3(-VL) rerank scoring&chat template (#16403) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-15 00:45:46 +08:00
shuwenn
48c2aca9ba [Env] centralize pd vars in environ.py (#16264) 2026-01-14 23:06:01 +08:00
fxmarty-amd
5af84c8af5 [AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392) 
Co-authored-by: Dehua Tang <dehtang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
 2026-01-14 01:44:40 -08:00
Yuan Luo
feae615b11 [VLM] Support ViT CUDA Graph for InternVL (#16732) 2026-01-14 17:29:23 +08:00
Netanel Haber
e75299a111 Fix issues/16714: Revert comment out of tl.debug_barrier() in causal_conv1d_triton (#16899) 
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2026-01-14 17:26:48 +08:00
roikoren755
72bacc88c8 [NemotronH] Use ReplicatedLinear for fc1_latent_proj (#16569) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-01-14 17:08:27 +08:00
shaharmor98
ba625c2d90 Feat/support nemotron h mtp (#17013) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
 2026-01-14 16:30:35 +08:00
shuwenn
cd33694585 feat: add --admin-api-key for finer-grained endpoint auth (#15908) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2026-01-13 20:21:55 -08:00
Liangsheng Yin
e2c8a50b38 fix grammar timeout sync across tp ranks. (#16898) 2026-01-14 10:26:31 +08:00
Lianmin Zheng
a4825ed588 Fix kernel type annotations for fp8 quant and logging (#16994) 2026-01-13 18:14:32 -08:00