chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
8,939 Commits 1 Branch 0 Tags
daa4841e867fe54495559e8259f4c9f6922d6a89
Commit Graph

4821 Commits

Author SHA1 Message Date
YAMY
daa4841e86 [ConfigArgumentMerger] Improve ConfigArgumentMerger compatibility with external callers (#17051) 2026-01-16 15:32:42 +08:00
Hudson Xing
21ee597e4a ci: enable offline mode when local cache is complete to avoid HF Hub … (#16121) 2026-01-15 20:15:33 -08:00
shuwenn
8ec160ed46 feature: support uvicorn access log filter(disable logging /metrics) (#15513) 2026-01-15 20:00:06 -08:00
Lianmin Zheng
e7dc85c50b Fix grammar sync across TP ranks (#17100) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-01-15 18:38:01 -08:00
PiteXChen
8b22deef5b fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 15:14:38 -08:00
JinYan Su
72e2f70ef7 feat(hicache): support numa detect to reduce long tail latency (#11028) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 14:11:49 -08:00
huangtingwei
77fc4c4a53 Add mooncake store read/write bandwidth logs (#10598) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Lei Huang <lei.huang@novita.ai>
 2026-01-15 12:15:41 -08:00
sglang-bot
000ad42225 chore: bump sgl-kernel version to 0.3.21 (#17075) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-01-15 12:41:17 +08:00
Hanming Lu
9d5f16d456 [SWA] fix swa radix cache match_len_since_tombstone update when hits swa_tombstone (#17061) 2026-01-15 11:28:33 +08:00
Ke Bao
7f8a58fffb Refactor prefix cache type checking (#17028) 2026-01-15 11:28:13 +08:00
b8zhong
4346db5faf [Fix] Remove assertion for padding for NVFP4 weight scales to fix GLM 4.5 NVFP4 (#12497) 2026-01-15 08:57:14 +08:00
Артем Савкин
424a380077 [NPU] NPU quantization refactoring & more quantization formats support (#14504) 
Co-authored-by: TamirBaydasov <mr.jeijy@gmail.com>
Co-authored-by: Tamir Baydasov <41994229+TamirBaydasov@users.noreply.github.com>
Co-authored-by: Савкин Артем <savkinartem@MacBook-Air-Viktoria.local>
Co-authored-by: Edward Shogulin <edward.shogulin@gmail.com>
 2026-01-15 04:25:15 +08:00
Simo Lin
f091858304 [gRPC] Add GetLoads RPC for comprehensive load metrics (#17087) 2026-01-14 12:05:10 -08:00
Aurick Qiao
5b1215d9da fix session request with None tokenizer (#16278) 2026-01-14 10:41:08 -08:00
Simo Lin
b3a3f51320 [API] Add /v1/loads endpoint for load metrics (#16976) 2026-01-14 09:13:20 -08:00
shuwenn
de94d793ad feat: support qwen3(-VL) rerank scoring&chat template (#16403) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-15 00:45:46 +08:00
shuwenn
48c2aca9ba [Env] centralize pd vars in environ.py (#16264) 2026-01-14 23:06:01 +08:00
fxmarty-amd
5af84c8af5 [AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392) 
Co-authored-by: Dehua Tang <dehtang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
 2026-01-14 01:44:40 -08:00
Yuan Luo
feae615b11 [VLM] Support ViT CUDA Graph for InternVL (#16732) 2026-01-14 17:29:23 +08:00
Netanel Haber
e75299a111 Fix issues/16714: Revert comment out of tl.debug_barrier() in causal_conv1d_triton (#16899) 
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2026-01-14 17:26:48 +08:00
roikoren755
72bacc88c8 [NemotronH] Use ReplicatedLinear for fc1_latent_proj (#16569) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-01-14 17:08:27 +08:00
shaharmor98
ba625c2d90 Feat/support nemotron h mtp (#17013) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
 2026-01-14 16:30:35 +08:00
shuwenn
cd33694585 feat: add --admin-api-key for finer-grained endpoint auth (#15908) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2026-01-13 20:21:55 -08:00
Liangsheng Yin
e2c8a50b38 fix grammar timeout sync across tp ranks. (#16898) 2026-01-14 10:26:31 +08:00
Lianmin Zheng
a4825ed588 Fix kernel type annotations for fp8 quant and logging (#16994) 2026-01-13 18:14:32 -08:00
Hubert Lu
afe285f7bd [AMD] enable CUDA graph for NSA backend and fix NSA FP8 fused RMSNorm group quant (#16841) 
Co-authored-by: wufann <715544327@qq.com>
 2026-01-13 17:36:01 -08:00
Byron Hsu
339915ce2b [logprob] Fix logprob + streaming for long concurrent decode by caching already processed logprob (#17005) 
Co-authored-by: root <root@memx-cge-29-sr1.xpop.twttr.net>
 2026-01-13 12:39:14 -08:00
Lianmin Zheng
075c5a5789 Code clean up for fp8 quantization (#16982) 2026-01-13 12:38:39 -08:00
James
ae0baefb94 [NPU] upgrade npu mf_apater plugin (#15853) 2026-01-13 09:02:10 +08:00
Yinghai Lu
d3c08fb07c [layers] support zero-dim rmsnorm (#16978) 2026-01-12 15:53:19 -08:00
Gaoji Liu
7b682de870 [Model] Support IQuest-Coder-40B-Loop (#16348) 
Co-authored-by: yxing <yxing@iquestlab.com>
Co-authored-by: yzhu <yzhu@ubiquant.com>
Co-authored-by: zelong518 <zelonghuang02@gmail.com>
 2026-01-12 23:44:45 +08:00
chenxu214
53dca74f47 Bugfix: EagleDraftWorker has not attribute "eagle_use_aux_hidden_state" (#16480) 2026-01-12 20:14:35 +08:00
Xun Sun
9f5cd80a8d Re-introduce the unit test of test_mooncake_ep_small (#16019) 2026-01-12 17:01:24 +08:00
Liangsheng Yin
5b7bed7ca4 Decouple grammar logic out of scheduler. (#16820) 2026-01-12 10:52:42 +08:00
chenxu140
38a88479c6 llama model and llama eagle3 model support dp-attn (#15268) 2026-01-12 08:54:56 +08:00
Yibo Cai
2f4a6addf3 [cpu/arm64] support run sglang on arm64 cpu (#14867) 2026-01-11 04:27:19 -08:00
Liangsheng Yin
09e2571e2e Clarify the meaning of cpu_group / entry_rank when dp + tp is enabled. (#16876) 2026-01-11 13:04:43 +08:00
Ratish P
c0248d6f37 [dpc]: unify DP controller load balancing and simplify dispatch logic (#16258) 2026-01-11 12:38:03 +08:00
Leoyzen
cf14feba4d Fix parallel tool call parsing bug when tool parameters contain arrays (#16345) 2026-01-10 19:31:38 -08:00
Mohammad Miadh Angkad
ff97814232 Tiny fix hicache kernel backend comparison (#16867) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
 2026-01-10 18:10:42 -08:00
DarkSharpness
d112f6a25b [Feature] Support JIT set kv cache (#16273) 2026-01-10 17:34:09 -08:00
Minglei Zhu
a2c2c09d7d [BugFix] fix gpt-oss-120b launch failure with --enable-piecewise-cuda-graph (#16757) 2026-01-11 09:19:59 +08:00
Yuwei An
2a9344d320 [tiny remove] remove torch_compile in parallel_state (#16865) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2026-01-10 16:04:25 -08:00
Qiaolin Yu
206db66f5c tiny refactor pcg split op registration (#16863) 2026-01-10 23:45:28 +08:00
Yuan Luo
d1ec93e3ac Optimize layernorm_gated for Qwen3-Next (#16397) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-10 20:55:31 +08:00
fzyzcjy
dae6a4092a Tiny add scheduler status logging (#16872) 2026-01-10 20:12:24 +08:00
fzyzcjy
a0899bdbd8 Fix log_decode_stats_every_iteration when having TP in attention (#16871) 2026-01-10 20:10:23 +08:00
fzyzcjy
641830c1c2 Tiny extract file logging utils (#16870) 2026-01-10 20:02:37 +08:00
YAMY
3fd88ea9b5 [MTP][spec_v2] Fix TRTLLM MLA backend crash in EAGLE draft_extend mode (#15790) 2026-01-10 19:58:23 +08:00
Yuwei An
145bd54f1b Piecewise Cuda Graph Memory Usage (#15927) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2026-01-10 19:29:13 +08:00