chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,184 Commits 1 Branch 0 Tags
432ecf841e261b6dcaacdf250a8691390b86558e
Commit Graph

45 Commits

Author SHA1 Message Date
YAMY
98b38de3f2 Fix: Safe RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads (#11871) 2025-11-25 01:19:06 +08:00
Liangsheng Yin
b2f7b08c49 Refactor cache init logic (#13800) 2025-11-24 11:41:46 +08:00
Liangsheng Yin
3e804bb0ba Update mem checker during busy (#13704) 2025-11-22 21:15:02 +08:00
Baizhou Zhang
8bfce9b08d [Tiny] Renaming environ for NVFP4 dispatch (#13756) 2025-11-22 00:05:20 -08:00
StonyPort
48ca9f7518 feat: support external custom models (#13429) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-20 11:16:58 +08:00
ykwd
075ba74dd4 [Doc] Update HiCache and Mooncake docs & Mooncake Setup Error Checking (#12740) 2025-11-19 12:03:27 +08:00
Zilin Zhu
9edb0e0da3 Add SGLANG_ENABLE_REQ_POOL_LEAK_STRICT_CHECK to bypass mem leak check (#13339) 2025-11-16 22:02:22 +08:00
Zilin Zhu
f0b5ccf5f5 [RL] Allow bypassing /health check (#13320) 2025-11-15 16:33:54 +08:00
Kaixi Hou
5ae0ac4244 [NVIDIA] Fix use case of SGLANG_ENABLE_FLASHINFER_GEMM (#13274) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-11-14 12:51:11 -08:00
billishyahao
ee3e337c62 [feat] make warmup timeout configurable through SGLANG_WARMUP_TIMEOUT (#13243) 2025-11-14 10:14:37 +08:00
Lianmin Zheng
19f6a33ce9 [Auto Sync] Update pynccl_wrapper.py, environ.py, registry.... (20251111) (#13097) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-13 15:58:47 -08:00
Zhihao Zhang
9c1c5c6d7d [ngram] use SGLANG_NGRAM_FORCE_GREEDY_VERIFY to control verify method (#13153) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-11-12 21:07:33 +08:00
vipwangerxiao
8f01a12d43 Improve overlap scheduling for better TTFT (#11856) 
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
 2025-11-12 11:24:04 +08:00
Liangsheng Yin
f09eee036d Tiny simplify evcition metrics collector (#12983) 2025-11-11 23:23:48 +08:00
Atream
ddd1440d0f Refactor KTransformers heterogeneous compute with unified GPU-quantization backend (#12834) 
Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com>
Co-authored-by: chenht2022 <cht22@mails.tsinghua.edu.cn>
Co-authored-by: skqliao <skqliao@gmail.com>
Co-authored-by: ovowei <1913953267@qq.com>
 2025-11-09 21:06:32 -08:00
Teng Ma
b8ac4fcb51 [PD] feat: refactor custom mem pool and add barex pd support (#12332) 2025-11-09 18:51:34 +08:00
hzh0425
ed5e905c89 [HotFix]: Add missing SGLANG_EPLB_HEATMAP_COLLECTION_INTERVAL env var (#12776) 2025-11-06 18:58:44 -08:00
Shu Wang
82f39dc11d Add mm_fp4 trtllm backend (#12406) 2025-11-05 14:31:46 -08:00
Atream
627bac649c Support Expert Deferral Mechanism in KTransformers (#12586) 
Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com>
Co-authored-by: chenht2022 <cht22@mails.tsinghua.edu.cn>
 2025-11-05 13:41:52 -08:00
zejunchen-zejun
0648eb482d [Profiler] Add SGLANG_PROFILE_RECORD_SHAPES for recording shapes when profiling (#11641) 
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2025-11-04 23:41:46 -08:00
Zhao Chen
d5fa019c36 feat: limit peak memory usage when computing logprobs (#6318) 
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-11-03 23:53:20 -08:00
Lianmin Zheng
20315697f4 move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521) 2025-11-02 13:15:05 -08:00
Xinyuan Tong
d2a8f71c2f [feat] Add SGLANG_TOOL_STRICT_LEVEL for tool-call behavior control (#12423) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-01 13:15:02 -07:00
Surya-Gunukula
795e98f8a6 Forward unknown tool calls instead of dropping (#12226) 2025-11-01 02:10:35 +00:00
Yuhong Guo
caa5d2967c feat: return partial generation results when aborting requests in waiting queue (#11673) 2025-10-29 22:03:00 +08:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
fzyzcjy
691c8534cf Support releasing CUDA graph memory when paused (#7873) 
Co-authored-by: ryang-max <y1cunhui.yang@gmail.com>
Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
 2025-10-28 14:40:50 +08:00
Lifu Huang
ce832d7034 Add env var to control custom Triton kernel cache and set CSGMV as default backend. (#12176) 2025-10-27 17:49:32 -07:00
Liangsheng Yin
8491c794ad [misc] depdencies & enviroment flag (#12113) 2025-10-26 14:52:35 +08:00
Liangsheng Yin
bda3758fac [log] Make forward iter count optional (#12116) 2025-10-26 14:51:07 +08:00
cctry
b0b4f71679 [Fix] memory leak by overlap + retract (#11981) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-10-23 22:59:23 +08:00
jacky.cheng
99c92ff24b [AMD] Support a new flag to disable quant on parallelLinear layer if required (#11811) 2025-10-22 19:16:15 -07:00
Christian Bahls
164302c7df Implement BGE-M3 Sparse Embeddings in SGLang (#10869) 
Co-authored-by: Christian Bahls <christian.bahls@planet-ai.de>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-22 13:46:16 -07:00
Atream
7e6191c098 init support for KTransformers Heterogeneous Computing (#11487) 
Co-authored-by: Jianwei Dong <1913953267@qq.com>
 2025-10-21 00:17:02 -07:00
Liangsheng Yin
57e25de756 Revert "Fix: Dynamic RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads" (#11827) 2025-10-19 19:44:06 +08:00
YAMY
80407b0493 Fix: Dynamic RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads (#10788) 2025-10-19 11:37:43 +08:00
Lianmin Zheng
9eefe2c0b7 Set CUDA_VISIBLE_DEVICES to achieve one GPU per process (#9170) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-10-17 17:30:06 -07:00
Liangsheng Yin
d88ac9bc9a [overlap-spec] Make plan stream an option (#11724) 2025-10-17 15:48:57 +08:00
StonyPort
fd389df96e Reduce the image processing latency in VLM (#11541) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
 2025-10-16 15:00:03 -07:00
Lianmin Zheng
cd7e1bd591 Sync code and test CI; rename some env vars (#11686) 2025-10-15 18:37:03 -07:00
Liangsheng Yin
acc2327bbd Move deep gemm related arguments to sglang.srt.environ (#11547) 2025-10-14 00:34:35 +08:00
Liangsheng Yin
f49419061d Move args from global_config to environ (#11332) 2025-10-12 21:29:31 +08:00
Liangsheng Yin
c882b5ae75 [CI] improve disaggregation CI. (#11264) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-10-08 21:40:56 +08:00
shubham singhal
03def5e3b1 Fix [test]: Env:SGLANG_TORCH_PROFILER_DIR for pytest. (#10780) 2025-10-03 22:59:32 +08:00
Liangsheng Yin
4a762041d7 move environ into sglang.srt to avoid break SRT auto sync. (#10791) 2025-09-23 02:04:20 -07:00