chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
6,536 Commits 1 Branch 0 Tags
fb2e816e83d7bf95ad439ee41bf10df774ec0023
Commit Graph

1160 Commits

Author SHA1 Message Date
Baizhou Zhang
7c45b8b4bb [CI] Fix qwen3-vl lora nightly ci (#12708) 2025-11-05 11:00:13 -08:00
Yuhong Guo
4d84f886e7 Refactor --debug-tensor-dump-layers to list (#12691) 2025-11-05 03:30:01 -08:00
Hubert Lu
3694266051 Expand and update test coverage for AMD CI (#10044) 2025-11-04 22:15:13 -08:00
Kangyan-Zhou
6dade6c3b5 Fix VLLM dependency test (#12670) 2025-11-04 20:49:58 -08:00
Liangsheng Yin
44b1b394a4 [PD-Disagg] Check finish after pop tranferred (#12638) 2025-11-05 11:18:09 +08:00
Kaixi Hou
0711d1509b [NVIDIA] Fix cutedsl backend of MoE (#12353) 2025-11-04 18:54:55 -08:00
soaringk
44da737770 [fix] Handle escaped characters in GLM tool call parser to prevent double serialization (#12456) 2025-11-04 16:48:14 -08:00
Baizhou Zhang
d22d044734 Revert "Enable memory saver for hybrid model" (#12648) 2025-11-04 16:22:06 -08:00
Liangsheng Yin
30b26ee9d0 Add io struct naming check back (#12634) 2025-11-05 01:15:01 +08:00
Liangsheng Yin
aa797d013d [Test] Merge all constrained decoding tests. (#12633) 2025-11-05 00:43:06 +08:00
fzyzcjy
b7d7041190 Add sanity checks when a test file is not added to CI (reland) (#12594) 2025-11-04 18:04:26 +08:00
Minglei Zhu
c14cc47e39 [Deterministic] Optimize bmm_batch_invariant op (#12522) 2025-11-04 00:33:31 -08:00
Zhao Chen
d5fa019c36 feat: limit peak memory usage when computing logprobs (#6318) 
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-11-03 23:53:20 -08:00
Junrong Lin
173e0f704f Enable memory saver for hybrid model (#11974) 2025-11-04 14:55:26 +08:00
Jonah Bernard
a209fb05c1 [Qwen3 VL] Add LoRA support for Qwen 3 VL (#12165) 2025-11-03 20:32:54 -08:00
kousakawang
7efd8b3d1f [FEAT] Shared mem pool based cuda ipc for multi-modal data transport (#11917) 
Co-authored-by: kousakawang <wanghanpei@bytedance.com>
Co-authored-by: Yuan Luo <4908075+yuan-luo@users.noreply.github.com>
 2025-11-02 16:46:37 +08:00
Baizhou Zhang
9a512cf95b [CI] Move some Lora/Deterministic CI tests to nightly (#12507) 2025-11-01 19:54:22 -07:00
Binyao Jiang
3451fc3280 [Feature] Qwen3-Next & FLA: Support MTP topk>1; Up to 6% faster (#11133) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-01 19:47:56 -07:00
Minglei Zhu
229256c505 [Deterministic] add deepseek v3 deterministic inference CI test (#12412) 2025-11-01 18:10:32 -07:00
Surya-Gunukula
795e98f8a6 Forward unknown tool calls instead of dropping (#12226) 2025-11-01 02:10:35 +00:00
Ke Bao
a4bf5c6ad2 Support Kimi Linear (#12469) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-10-31 14:03:35 -07:00
Yuan Luo
c30ebb9300 [VLM] Optimize async mm data process mechanism (#12066) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-01 01:24:53 +08:00
Neelabh Sinha
b57dc169eb [Test] Add Functional Tests for Penalty Parameters (#11931) 2025-11-01 00:10:17 +08:00
Yuhong Guo
ab95d35fcb feat: Add Non-intrusive Tensor Dumping for Model Inference (#10566) 2025-10-31 12:04:48 +08:00
Liangsheng Yin
d4a09ec9dc [CI] fix tests' time estimation (#12401) 2025-10-30 19:20:35 -07:00
Kaixi Hou
c0d02cf4d1 [NVIDIA] Add CI workloads for GB200 (#12242) 2025-10-30 14:32:03 -07:00
Even Zhou
ce6b17c0f9 [Feature] Support DeepSeek MTP on NPU (#11897) 
Co-authored-by: liupeng374 <liupeng374@huawei.com>
 2025-10-30 21:56:56 +08:00
Liangsheng Yin
73dfd2dfb1 [Test] Enhance radix cache test for spec cases (#12394) 2025-10-30 18:46:28 +08:00
Kangyan-Zhou
7e28c67d19 Fix DeepSeek chat templates to handle tool call arguments type checking (#11700) (#12123) 2025-10-30 16:39:25 +08:00
Baizhou Zhang
621dfb8886 Import flash_mla from sgl-kernel (#12135) 2025-10-29 23:54:21 -07:00
b8zhong
7ed8ba05cb [CI] Add Llama 3.1 8B FP4 to B200 CI (#12182) 2025-10-29 14:56:41 -07:00
Minglei Zhu
e39628fd07 [2/2] Deepseek deterministic: support deepseek v3 deterministic inference on 8 x H200 (#12095) 2025-10-29 11:49:04 -07:00
Yuhong Guo
caa5d2967c feat: return partial generation results when aborting requests in waiting queue (#11673) 2025-10-29 22:03:00 +08:00
Rain H
750940ae36 Eagle3 DP attention for Qwen3 MoE (#12002) 2025-10-29 20:25:17 +08:00
Baizhou Zhang
685c06451f [ci] Try fixing broken CIs (#12317) 2025-10-29 01:13:51 -07:00
Xiaoyu Zhang
334543ff3b Add continuous_usage_stats support for streaming responses (#12241) 2025-10-29 10:01:23 +08:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
Chenxi Li
cc7b04a29c Feature/Add GET endpoint to query loaded LoRA adapters (#12229) 2025-10-28 01:22:00 -07:00
fzyzcjy
691c8534cf Support releasing CUDA graph memory when paused (#7873) 
Co-authored-by: ryang-max <y1cunhui.yang@gmail.com>
Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
 2025-10-28 14:40:50 +08:00
Lifu Huang
ce832d7034 Add env var to control custom Triton kernel cache and set CSGMV as default backend. (#12176) 2025-10-27 17:49:32 -07:00
Yineng Zhang
a6ea3add76 [Auto Sync] Update scheduler.py, spec_info.py, run_suite.py... (20251027) (#12235) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>
 2025-10-27 17:21:08 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
Kangyan-Zhou
da1766e444 Remove deprecated --enable-beta-spec argument and fix b200 test (#12167) 
Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-26 15:54:40 -07:00
Zijian Zhang
a8023891f6 model: support NVILA and NVILA Lite (#10399) 2025-10-26 09:58:09 -07:00
fzyzcjy
0103f374ba Support DeepGEMM for deterministic inference (#12142) 2025-10-26 22:36:17 +08:00
Liangsheng Yin
5994e6c373 Do not use MagicMock to mock server_args in tests (#12154) 2025-10-26 16:00:46 +08:00
Kai-Hsun Chen
6371f7af27 [quantization] AWQ Marlin doesn't work when dtype is bfloat16 (#11494) 
Signed-off-by: Kai-Hsun Chen <khchen@x.ai>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-10-26 15:49:45 +08:00
Liangsheng Yin
8491c794ad [misc] depdencies & enviroment flag (#12113) 2025-10-26 14:52:35 +08:00
Johnsonms
b72f9f08c1 [Test] Add dsv3.2 nsa backend testing (#11936) 2025-10-25 17:07:37 -07:00