chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
9,885 Commits 1 Branch 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph

9885 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yuan Luo
21ec99beff [VLM][Doc] Document for VLM DP Encoder (#14279) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 15:08:37 +08:00
Simo Lin
383689e3ad [model-gateway] fix version output (#14276) 2025-12-01 22:51:57 -08:00
b8zhong
236a7c2370 fix trtllm mla spec (#13738) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-01 22:16:25 -08:00
Roger Young
3dabd609fb Optimize topk sigmoid in minimax_m2 (#14047) 
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-12-02 14:07:12 +08:00
Simo Lin
73df525382 [model-gateway] include smg version command in py binding (#14274) 2025-12-01 21:21:02 -08:00
b8zhong
e6420100ee sync attention doc and ep doc to doctree (#14257) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-01 21:15:22 -08:00
Kevin Li
c9e2090101 fix: Support PP for Mistral Small 3.1 (#14254) 2025-12-02 13:04:14 +08:00
kun-llfl
106df4eac5 Fix mrope_positions size when req is retracted (#13700) 
Signed-off-by: Kun(llfl) <i@imux.top>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-02 11:38:20 +08:00
Liangsheng Yin
427a19b643 Remove cargo config also in .zshenv (#14267) 2025-12-02 11:31:53 +08:00
Mick
1f930cd23d [diffusion] CI: add testcase-wise retry mechanism (#14261) 2025-12-02 11:06:12 +08:00
Kartik Ramesh
11ce05163d Fix NIXL exception message (#14172) 2025-12-02 10:39:45 +08:00
Simo Lin
cd4151abc7 [model-gateway] add audio and moderation in model card (#14263) 2025-12-01 18:36:38 -08:00
Stefan He
8fe8b63576 Revert "Try to remove wrong logic about max total token in spec decoding" (#14259) 2025-12-01 18:18:03 -08:00
Lianmin Zheng
8a7b1b8301 [Docs] Update CI docs (#14260) 2025-12-01 18:15:03 -08:00
Yuan Luo
26aebf83d3 [VLM] Support Piecewise CUDA Graph for Qwen3-Omni-MOE (#14222) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 10:12:10 +08:00
Mick
3ab8ae6847 [diffusion] fix: fix Flux.2 condition image resize (#14232) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-02 10:05:44 +08:00
Baizhou Zhang
03888b9de5 [Minor] Upgrade cutedsl version in Dockerfile (#13968) 2025-12-01 17:15:26 -08:00
Lianmin Zheng
796d82b107 [Auto Sync] Add max_total_num_tokens metric: Update scheduler_metrics_mixin.py, collector.py (20251202) (#14256) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dan Zheng <dzheng@x.ai>
 2025-12-01 16:34:34 -08:00
Lianmin Zheng
1da59e8304 [Auto Sync] optionally disable fake register in Update fp8_kernel.py (20251202) (#14255) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gauravjain14 <41287729+gauravjain14@users.noreply.github.com>
 2025-12-01 16:11:12 -08:00
Xinyue Zhang
1d66a14c2e [model-gateway] Add e2e tests of streaming events and tool choice for response api (#13880) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2025-12-01 15:27:12 -08:00
TomerBN-Nvidia
02af51e4fc Support fp4 fp8 non gated moe (#13794) 
Co-authored-by: Roi Koren <roik@nvidia.com>
Co-authored-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
 2025-12-01 15:26:28 -08:00
Baizhou Zhang
eb5008846a [CI] Fix test_deepep_large.py (#14247) 2025-12-01 15:18:48 -08:00
Zhiyu
079b173853 Fix a distributed initialization error (#13843) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-12-01 15:10:05 -08:00
Simo Lin
57f933fd7d [model-gateway] Migrate Worker trait to model-aware methods (#14250) 2025-12-01 14:32:09 -08:00
YAMY
1f2b84d28d Fix NSA Bug in Centralize NSA Dispatch Logic (#14245) 2025-12-01 13:18:18 -08:00
Simo Lin
e7d6027e4a [model-gateway] add ModelCard support to WorkerMetadata (#14243) 2025-12-01 13:04:09 -08:00
ishandhanani
07821352fb Revert "Skip weight loading in deepgemm compilation" (#14241) 2025-12-01 12:59:09 -08:00
Byron Hsu
edbeaf3b88 [MM][style] rename inputs_embeds to input_embeds for consistency (#14240) 2025-12-01 11:36:51 -08:00
Kangyan-Zhou
d9dca28247 Update pr-test.yml to fix unknown job name deepep-8-gpu 2025-12-01 10:02:29 -08:00
Simo Lin
9325f94534 [model-gateway] add ModelCard and ProviderType for model configuration (#14237) 2025-12-01 09:47:51 -08:00
Kangyan-Zhou
41b7aab848 Disable Deepep 8 GPU tests (#14152) 2025-12-01 09:37:38 -08:00
Sulfur6-L8972
491f4fe8e1 Modify git tag for DeepGemm in sgl-kernel. (#14179) 2025-12-01 09:01:22 -08:00
sglang-bot
34035d8cd9 chore: bump sgl-kernel version to 0.3.18.post2 (#14229) 2025-12-02 00:50:40 +08:00
Simo Lin
92ca629582 [model-gateway] add ModelType bitflags and Endpoint enum for worker (#14230) 2025-12-01 08:14:44 -08:00
Chang Su
ec92d7f14e [model-gateway] fix v1/models response format to be oai compatible (#13693) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2025-12-01 07:20:38 -08:00
Simo Lin
79b389da6b [model-gateway] refactor oai router 1/n (#14228) 2025-12-01 07:16:35 -08:00
Xiaoyu Zhang
3de09aadbc Add new moe wna16 marlin gemm (#14122) 2025-12-01 23:07:53 +08:00
liupeng374
2e8f54e61e [spec-overlap] bugfix for pd disaggregation and npu (#14088) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-12-01 22:58:20 +08:00
Wenyi Xu
e55731b6e8 [model-gateway] Avoid logging MCP connection token (#13887) 2025-12-01 04:44:31 -08:00
fzyzcjy
45264554f3 Super tiny fix typo (#14219) 2025-12-01 20:19:17 +08:00
Lianmin Zheng
c4293f59ac Change PR test schedule to run every 6 hours (#14218) 2025-12-01 03:50:58 -08:00
Liangsheng Yin
a2423052f6 Add cuda event based on waiting value (#14214) 2025-12-01 18:51:44 +08:00
Lianmin Zheng
bc3d2a85af [Minor] update docs (#14212) 2025-12-01 02:33:58 -08:00
fzyzcjy
d815d00248 Tiny call cudaProfilerStart only on first rank in node (#14211) 2025-12-01 18:18:45 +08:00
Xiaoyu Zhang
fa9021b21f fix: Increase FlashInfer workspace size for Qwen3VL models (#14173) 2025-12-01 17:54:23 +08:00
Xiaoyu Zhang
9c80072845 Add peak output tokens per second in bench_serving (#14165) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-01 17:47:54 +08:00
Yuan Luo
630a693081 [VLM] Boost Memory Pool based CUDA IPC (#14123) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-01 17:17:46 +08:00
Mick
7ce8faae28 [diffusion] refactor: remove hard-code of instanceof on PipelineConfig (#14186) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-01 16:35:34 +08:00
fzyzcjy
de153cf76a Fix speculative decoding error when retracting (#14180) 2025-12-01 15:30:13 +08:00
fzyzcjy
f4a0c5c76b Try to remove wrong logic about max total token in spec decoding (#14167) 2025-12-01 15:29:58 +08:00