chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
6,536 Commits 1 Branch 0 Tags
fb2e816e83d7bf95ad439ee41bf10df774ec0023
Commit Graph

778 Commits

Author SHA1 Message Date
zejunchen-zejun
0648eb482d [Profiler] Add SGLANG_PROFILE_RECORD_SHAPES for recording shapes when profiling (#11641) 
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2025-11-04 23:41:46 -08:00
sglang-bot
09938e1f82 chore: bump SGLang version to 0.5.4.post3 (#12639) 2025-11-04 18:32:11 -08:00
Trevor Morris
dbcf85b7f0 Add --speculative-moe-runner-backend server arg (#10183) 2025-11-04 00:20:56 -08:00
Teng Ma
1689c0e35f [Doc] fix miss index for production request trace (#12547) 2025-11-03 17:57:09 -08:00
b8zhong
d31d48b341 update usage of trtllm_fp8_per_tensor_scale_moe (#12569) 2025-11-03 14:25:32 -08:00
Kangyan-Zhou
6a3b9fd00f Update setup_github_runner.md 2025-11-02 20:44:09 -08:00
Xinyuan Tong
d2a8f71c2f [feat] Add SGLANG_TOOL_STRICT_LEVEL for tool-call behavior control (#12423) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-01 13:15:02 -07:00
Surya-Gunukula
795e98f8a6 Forward unknown tool calls instead of dropping (#12226) 2025-11-01 02:10:35 +00:00
sglang-bot
41c10e67fc chore: bump SGLang version to 0.5.4.post2 (#12439) 2025-10-31 17:38:50 -07:00
yinghui
a80bcb5a68 Add env var to disable FA4 warmup (#12430) 2025-10-31 12:25:00 -07:00
b8zhong
a076ec1a7a Revert "fix llama4 kv cache layout" (#12437) 2025-10-30 22:33:37 -07:00
Lianmin Zheng
7cd716f776 Update news section in README.md (#12409) 2025-10-30 12:41:21 -07:00
Baizhou Zhang
621dfb8886 Import flash_mla from sgl-kernel (#12135) 2025-10-29 23:54:21 -07:00
Teng Ma
32438eba45 [Ckpt Engine] feat: new sglang entrypoint support for update (#12216) 2025-10-30 10:39:27 +08:00
Yuhong Guo
caa5d2967c feat: return partial generation results when aborting requests in waiting queue (#11673) 2025-10-29 22:03:00 +08:00
Yingchun Lai
5e36a0b455 [metrics][EPLB]: Support selected count of physical experts on each GPU (#9825) 2025-10-28 20:56:19 -07:00
hlu1
0ee831dee0 Update deepseek_v32.md (#12296) 2025-10-28 14:52:38 -07:00
ybyang
9c6e25d2a6 doc for logit_bias (#12188) 2025-10-28 10:32:12 -07:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
ishandhanani
285a8e6986 docker: add CUDA13 support in dockerfile and update GDRCopy/NVSHMEM for blackwell support (#11517) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-27 22:00:54 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
sglang-bot
55d75e11bd chore: bump SGLang version to 0.5.4.post1 (#12169) 2025-10-27 09:35:20 +08:00
Xinyuan Tong
3f4cc0aff0 Remove description for --enable-beta-spec argument (#12177) 2025-10-26 16:49:39 -07:00
赵晨阳
7ebc28f5d6 [WIP] support MiniMax M2 model (#12129) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Roger Young <42564206+rogeryoungh@users.noreply.github.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-10-26 13:58:54 -07:00
Zijian Zhang
a8023891f6 model: support NVILA and NVILA Lite (#10399) 2025-10-26 09:58:09 -07:00
Baizhou Zhang
97828878d8 [Doc] Small update of DeepSeek v3.2 document (#12138) 2025-10-25 20:34:05 -07:00
Vincent Zhong
b4d2da106e [docs] upd docker files names everywhere (#12133) 2025-10-25 18:20:23 -07:00
Baizhou Zhang
bcecf27e7c [Doc] Fix format for deepseek v3.2 document (#12130) 2025-10-25 15:07:50 -07:00
Baizhou Zhang
8e987fa2a3 Update document index for DeepSeek-v32 docs (#12101) 2025-10-25 13:38:58 -07:00
Lianmin Zheng
4caca1ba04 Clean up server args & Add CI scripts (#12124) 2025-10-25 11:53:57 -07:00
Kangyan-Zhou
ceb105a780 Update setup_github_runner.md 2025-10-25 09:22:00 -07:00
Baizhou Zhang
729b242934 [Doc] Add documentation for DeepSeek V3.2 (#11877) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
 2025-10-24 19:06:22 -07:00
Jonah Bernard
4b046a72d3 docs(server-arguments): add allowed options for each argument (#11560) 2025-10-24 11:49:20 -07:00
ishandhanani
14203432b4 fix(compile_utils, ep_moe): update environment variable and dtype check (#12034) 2025-10-24 11:00:12 -07:00
Lianmin Zheng
ffc722a690 Revert "lang: support direct video inference" (#12038) 2025-10-23 19:21:31 -07:00
thelongestusernameofall
49afb3d9d9 Fix(security): block unsafe pickle deserialization to mitigate CVE-2025-10164 (#11909) 
Co-authored-by: Chengxing Xie <xiechengxing34@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-23 19:12:40 -07:00
Mick
823b442945 lang: support direct video inference (#9936) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2025-10-23 18:12:39 -07:00
sglang-bot
1053e1be17 chore: bump SGLang version to 0.5.4 (#12027) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2025-10-23 18:01:40 -07:00
Teng Ma
96a5e4dd79 [Feature] Support loading weights from ckpt engine worker (#11755) 
Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
Co-authored-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-10-23 09:23:30 -07:00
Zaili Wang
007b849b0e [CPU] misc updates (#11906) 2025-10-22 21:10:05 -07:00
Baizhou Zhang
983ef22cf3 [Doc] Update deterministic inference flag in server_arguments.md (#11978) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-22 14:12:15 -07:00
Minglei Zhu
200a3c0bb1 [Documentation] add doc for deterministic inference (#11956) 2025-10-22 12:36:15 -05:00
Zhiyu
80b2b3207a Enable native ModelOpt quantization support (3/3) (#10154) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-10-21 21:44:29 -07:00
Baizhou Zhang
ef4a8097b8 Rename flashmla kernel options of nsa backend for better readability (#11876) 2025-10-21 13:14:16 -07:00
ybyang
dbb16bedd5 Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix #6572] (#11416) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: YorkSu <york_su@qq.com>
 2025-10-21 16:27:56 +08:00
Neelabh Sinha
852c0578fd [FEATURE] Add OpenAI-Compatible LoRA Adapter Selection (#11570) 2025-10-21 15:44:33 +08:00
Meng, Hengyu
b113c72e7a Init attention backend for Intel XPU (#10656) 
Co-authored-by: guangyey <guangye.yu@intel.com>
Co-authored-by: DiweiSun <105627594+DiweiSun@users.noreply.github.com>
 2025-10-21 11:41:28 +08:00
DarkSharpness
276e7b3e4e [Feature] New structural tag support (#10691) 2025-10-20 18:25:58 +08:00
Sai Enduri
e53bf44243 Update amd gpu install docs. (#11849) 2025-10-20 00:03:26 -07:00