chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
11,116 Commits 1 Branch 0 Tags
47cb9b9d116d1c5c112e894110ec94de80f2db6b
Commit Graph

11116 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sogalin
0da9845ef1 Modify rocm.Dockerfile (#12274) 2025-10-28 08:36:32 -07:00
Keyang Ru
9288544180 [router] Fix type unmatch during validation (#12257) 2025-10-28 06:04:54 -07:00
Yineng Zhang
64cf868eba chore: cleanup quant deps (#12268) 2025-10-28 02:03:57 -07:00
Yineng Zhang
ea39952797 Revert "[Feature] PD-Multiplexing Context and Scheduler." (#12267) 2025-10-28 02:00:37 -07:00
Shangming Cai
41a113356a Fix potential eos bug on decode instance when PD is enabled (#12206) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-10-28 01:29:02 -07:00
Xuchun Shang
a1f2dc90e4 [Bug fix] [PP] fix wrong dtype for quantified model (#12247) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-10-28 01:27:24 -07:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
Cheng Wan
b1e13e7cea [hotfix] Incorrect CombineOverlapArgs in SBO (#12230) 2025-10-28 01:23:06 -07:00
Chenxi Li
cc7b04a29c Feature/Add GET endpoint to query loaded LoRA adapters (#12229) 2025-10-28 01:22:00 -07:00
Simo Lin
d85d6dba3b [router] configure workflow retries and timeout based on routerConfig (#12252) 2025-10-28 00:41:20 -07:00
Simo Lin
c5642a7a7a [router] use mcp struct from sdk and clean up code across codebase (#12249) 2025-10-28 00:33:10 -07:00
fzyzcjy
691c8534cf Support releasing CUDA graph memory when paused (#7873) 
Co-authored-by: ryang-max <y1cunhui.yang@gmail.com>
Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
 2025-10-28 14:40:50 +08:00
Yongfei Xu
d2b8c4123e Opt fused triton moe: add tma for down proj kernel (#10567) 
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-10-28 14:26:17 +08:00
Scott Lee
bf8f7a944f Add per-request retraction count (#11177) 2025-10-27 23:22:34 -07:00
hlu1
81a632ace6 [DeepseekV32] Enable flashmla_prefill kernel with fp8 kvcache (#11655) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-10-27 23:11:48 -07:00
Simo Lin
83b2240074 [router] remove code duplication (#12245) 2025-10-27 23:01:44 -07:00
ishandhanani
285a8e6986 docker: add CUDA13 support in dockerfile and update GDRCopy/NVSHMEM for blackwell support (#11517) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-27 22:00:54 -07:00
Yuan Luo
813bd6f85c [2/2] Use moe_sum_reduce cuda kernel (#10654) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
 2025-10-28 12:01:57 +08:00
Xinyuan Tong
729f612dc6 Update openai package version to 2.6.1 (#12222) 2025-10-28 11:23:40 +08:00
jianan-gu
899453ac50 Use explicit uint64 dtype for Tensor data_ptr() to avoid overflow (#11994) 2025-10-27 19:05:57 -07:00
Lifu Huang
ce832d7034 Add env var to control custom Triton kernel cache and set CSGMV as default backend. (#12176) 2025-10-27 17:49:32 -07:00
weiliang
88596739a4 Support running FP4 Deepseek on SM120. (#11708) 2025-10-27 17:37:49 -07:00
Yineng Zhang
a6ea3add76 [Auto Sync] Update scheduler.py, spec_info.py, run_suite.py... (20251027) (#12235) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>
 2025-10-27 17:21:08 -07:00
fzyzcjy
326c84c493 Compiling rope while preserving true on policy (#12161) 2025-10-28 08:02:17 +08:00
gongwei-130
8da608cce0 fix: AttributeError: 'NixlKVManager' object has no attribute 'prefill_tp_size_table' (#12234) 2025-10-27 15:57:33 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Chunyuan WU
c11b34d599 rope xpu: fix missing argument 'fused_set_kv_buffer_arg' and replace native with sgl_kernel_xpu impl (#12006) 2025-10-28 01:02:18 +08:00
ykcombat
05ad28f25e [Feature] PD-Multiplexing Context and Scheduler. (#11592) 2025-10-28 00:54:43 +08:00
pansicheng
0cae873fcd check_offload_progress more frequently (#11656) 2025-10-28 00:37:38 +08:00
Haichao Zhu
a8b91f6b2d improve mimax-m2 rmsnorm precision (#12186) 2025-10-28 00:01:42 +08:00
Jimmy
959d1ab84b fix(metrics): double times add_latency for DECODE_BOOTSTRAP (#12209) 2025-10-27 23:59:48 +08:00
Muqi Li
6c1c193308 [Detokenizer Manager] Cleanup state when reqs are finished (#12205) 2025-10-27 23:59:09 +08:00
cctry
3029d30189 Fix crash after flush cache (#12107) 
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
 2025-10-27 23:52:27 +08:00
Yuan Luo
f389f01714 Optimize triton_mrope with torch compile (#12112) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-27 23:49:22 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
Yuxuan Zhang
a88b006ecf GLM-4-0414 and GLM-4.1V Code Refactor (#12117) 2025-10-27 16:57:07 +08:00
Qi Yuhang
ce112c07fe [sgl-kernel][4/N]Support Expert Specialization Grouped GEMM (#12080) 2025-10-27 01:20:01 -07:00
Joonchen Liau
f7dc2f334b [sgl-kernel] feat: Support sm120 cutlass fp8 gemm kernel (#9403) 2025-10-26 23:45:45 -07:00
Yineng Zhang
cd784faf20 docs: update contact (#12192) 2025-10-26 23:23:39 -07:00
Baizhou Zhang
75c09e1ffe [Fix] Fix cu130 sgl-kernel wheel renaming (#12173) 2025-10-26 22:44:05 -07:00
rongfu.leng
09af0a7b5a [sgl-route] Optimize the use of constant slices and retain to simplif… (#12159) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2025-10-26 21:14:33 -07:00
Kevin_Xiong
c8d385ce68 [doc] add example of using w4fp8 for Deepseek (#12057) 
Co-authored-by: Chao Xiong <chaoxiong@Chaos-MacBook-Air.local>
 2025-10-26 20:10:38 -07:00
sglang-bot
55d75e11bd chore: bump SGLang version to 0.5.4.post1 (#12169) 2025-10-27 09:35:20 +08:00
Xinyuan Tong
3f4cc0aff0 Remove description for --enable-beta-spec argument (#12177) 2025-10-26 16:49:39 -07:00
ishandhanani
cadfae666d fix broken deepep/flashmla install in container by adding --no-build-isolation (#12170) 2025-10-26 16:39:51 -07:00
Kangyan-Zhou
da1766e444 Remove deprecated --enable-beta-spec argument and fix b200 test (#12167) 
Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-26 15:54:40 -07:00
Chang Su
a124b517f6 [router] Remove SharedXxxStorage type aliases to make Arc explicit (#12171) 2025-10-26 14:48:07 -07:00
Chang Su
d05a968ba8 [router][grpc] Add ResponsesContext and fix error propagation in responses api (#12164) 2025-10-26 14:07:19 -07:00
Chang Su
94aad0de99 [misc][grpc] Remove duplicate log (#12168) 2025-10-26 14:06:59 -07:00
赵晨阳
7ebc28f5d6 [WIP] support MiniMax M2 model (#12129) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Roger Young <42564206+rogeryoungh@users.noreply.github.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-10-26 13:58:54 -07:00