chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
6,380 Commits 1 Branch 0 Tags
ecb9fa14e6baaedf255bfbc065e2a58a76efb1df
Commit Graph

1134 Commits

Author SHA1 Message Date
Even Zhou
ce6b17c0f9 [Feature] Support DeepSeek MTP on NPU (#11897) 
Co-authored-by: liupeng374 <liupeng374@huawei.com>
 2025-10-30 21:56:56 +08:00
Liangsheng Yin
73dfd2dfb1 [Test] Enhance radix cache test for spec cases (#12394) 2025-10-30 18:46:28 +08:00
Kangyan-Zhou
7e28c67d19 Fix DeepSeek chat templates to handle tool call arguments type checking (#11700) (#12123) 2025-10-30 16:39:25 +08:00
Baizhou Zhang
621dfb8886 Import flash_mla from sgl-kernel (#12135) 2025-10-29 23:54:21 -07:00
b8zhong
7ed8ba05cb [CI] Add Llama 3.1 8B FP4 to B200 CI (#12182) 2025-10-29 14:56:41 -07:00
Minglei Zhu
e39628fd07 [2/2] Deepseek deterministic: support deepseek v3 deterministic inference on 8 x H200 (#12095) 2025-10-29 11:49:04 -07:00
Yuhong Guo
caa5d2967c feat: return partial generation results when aborting requests in waiting queue (#11673) 2025-10-29 22:03:00 +08:00
Rain H
750940ae36 Eagle3 DP attention for Qwen3 MoE (#12002) 2025-10-29 20:25:17 +08:00
Baizhou Zhang
685c06451f [ci] Try fixing broken CIs (#12317) 2025-10-29 01:13:51 -07:00
Xiaoyu Zhang
334543ff3b Add continuous_usage_stats support for streaming responses (#12241) 2025-10-29 10:01:23 +08:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
Chenxi Li
cc7b04a29c Feature/Add GET endpoint to query loaded LoRA adapters (#12229) 2025-10-28 01:22:00 -07:00
fzyzcjy
691c8534cf Support releasing CUDA graph memory when paused (#7873) 
Co-authored-by: ryang-max <y1cunhui.yang@gmail.com>
Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
 2025-10-28 14:40:50 +08:00
Lifu Huang
ce832d7034 Add env var to control custom Triton kernel cache and set CSGMV as default backend. (#12176) 2025-10-27 17:49:32 -07:00
Yineng Zhang
a6ea3add76 [Auto Sync] Update scheduler.py, spec_info.py, run_suite.py... (20251027) (#12235) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>
 2025-10-27 17:21:08 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
Kangyan-Zhou
da1766e444 Remove deprecated --enable-beta-spec argument and fix b200 test (#12167) 
Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-26 15:54:40 -07:00
Zijian Zhang
a8023891f6 model: support NVILA and NVILA Lite (#10399) 2025-10-26 09:58:09 -07:00
fzyzcjy
0103f374ba Support DeepGEMM for deterministic inference (#12142) 2025-10-26 22:36:17 +08:00
Liangsheng Yin
5994e6c373 Do not use MagicMock to mock server_args in tests (#12154) 2025-10-26 16:00:46 +08:00
Kai-Hsun Chen
6371f7af27 [quantization] AWQ Marlin doesn't work when dtype is bfloat16 (#11494) 
Signed-off-by: Kai-Hsun Chen <khchen@x.ai>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-10-26 15:49:45 +08:00
Liangsheng Yin
8491c794ad [misc] depdencies & enviroment flag (#12113) 2025-10-26 14:52:35 +08:00
Johnsonms
b72f9f08c1 [Test] Add dsv3.2 nsa backend testing (#11936) 2025-10-25 17:07:37 -07:00
Qiaolin Yu
d98b81e260 Accelerate deepseek fp4 b200 ci (#11993) 2025-10-25 15:29:14 -07:00
Kangyan-Zhou
9e656dd3b2 Reenable b200 tests (#11814) 2025-10-25 13:02:33 -07:00
Lifu Huang
d9b31011aa Skip TestLlama4LoRA in CI (#12098) 2025-10-25 12:16:36 -07:00
Baizhou Zhang
ce86979355 [Fix] Set global args in cpu test (#12105) 2025-10-24 21:46:17 -07:00
fzyzcjy
20bd2271e2 Support true on-policy (#12058) 2025-10-25 10:23:42 +08:00
Jinwu
13bf565d60 [2/N]Support DeepSeek-R1 w4a8 low latency deepep (#8464) 
Co-authored-by: Hank Han <hanhan7630@outlook.com>
Co-authored-by: Shangchuan Huang <2510421000@qq.com>
 2025-10-24 17:41:16 -07:00
Glen Liu
fc86b18b3e adjust dynamic vs static outputs comparison in test_lora_update.py (#11884) 2025-10-24 10:35:34 -07:00
narutolhy
1801cd199f support more model in piecewise cuda graph (#11745) 2025-10-24 10:31:39 +08:00
Jonah Bernard
62eff37ba1 Refactor Triton-kernel MoE runner integration (#11795) 2025-10-23 18:47:28 -07:00
Fan Yin
14a4d80e57 [8/n] decouple quantization impl from vllm dependency - gguf srt (#11964) 
Co-authored-by: Peng Zhang <zhuangsen.zp@antgroup.com>
 2025-10-23 18:12:00 -07:00
b8zhong
8ae9d4bb41 Revert "[ROCm] Remove vLLM rope dependency & use AITER impl" (#12028) 2025-10-23 12:42:59 -07:00
Mick
770529a731 model: support deepseek-ocr (#11891) 
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Shi Shuai <126407087+shuaills@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-10-24 03:15:17 +08:00
Mick
8bd26dd4e6 ci: fix night-ci with push retry mechanism (#11765) 2025-10-23 11:31:05 -07:00
Netanel Haber
a98496834b Feature/nano v2 offline modelopt fp8 and nvfp4 (#12018) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2025-10-23 11:16:46 -07:00
cctry
b0b4f71679 [Fix] memory leak by overlap + retract (#11981) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-10-23 22:59:23 +08:00
Liangsheng Yin
6c18addb6f Revert "Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4" (#12015) 2025-10-23 21:27:58 +08:00
Netanel Haber
d6fee73d1f Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4 (#11866) 2025-10-23 17:29:02 +08:00
blzheng
13fb8b5489 [CPU] Optimize FP16 decode_attention_cpu (#10652) 2025-10-22 21:39:51 -07:00
b8zhong
4d4feccbb2 [ROCm] Remove vLLM rope dependency & use AITER impl (#11322) 2025-10-22 19:17:34 -07:00
Shangming Cai
7fceeef599 Fix flaky hicache test with mooncake backend (#11953) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-10-22 21:00:47 +08:00
Hank Han
904655c5fd [2/N] Added the core structure of elastic EP and the eplb algorithm with faulty rank (#10606) 
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-10-22 01:13:31 -07:00
Zhiyu
80b2b3207a Enable native ModelOpt quantization support (3/3) (#10154) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-10-21 21:44:29 -07:00
b8zhong
d0a64c7e2c vlm: enforce pybase64 for image and str encode/decode (#10700) 2025-10-21 19:05:32 +08:00
Shangming Cai
05d3667ab9 [CI] disable glm4.1v and fix the flashinfer installation (#11902) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-10-21 18:38:35 +08:00
ybyang
dbb16bedd5 Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix #6572] (#11416) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: YorkSu <york_su@qq.com>
 2025-10-21 16:27:56 +08:00
Neelabh Sinha
852c0578fd [FEATURE] Add OpenAI-Compatible LoRA Adapter Selection (#11570) 2025-10-21 15:44:33 +08:00