chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
6,380 Commits 1 Branch 0 Tags
ecb9fa14e6baaedf255bfbc065e2a58a76efb1df
Commit Graph

214 Commits

Author SHA1 Message Date
Yuhong Guo
caa5d2967c feat: return partial generation results when aborting requests in waiting queue (#11673) 2025-10-29 22:03:00 +08:00
Yingchun Lai
5e36a0b455 [metrics][EPLB]: Support selected count of physical experts on each GPU (#9825) 2025-10-28 20:56:19 -07:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
Baizhou Zhang
8e987fa2a3 Update document index for DeepSeek-v32 docs (#12101) 2025-10-25 13:38:58 -07:00
Baizhou Zhang
729b242934 [Doc] Add documentation for DeepSeek V3.2 (#11877) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
 2025-10-24 19:06:22 -07:00
ishandhanani
14203432b4 fix(compile_utils, ep_moe): update environment variable and dtype check (#12034) 2025-10-24 11:00:12 -07:00
Lianmin Zheng
ffc722a690 Revert "lang: support direct video inference" (#12038) 2025-10-23 19:21:31 -07:00
Mick
823b442945 lang: support direct video inference (#9936) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2025-10-23 18:12:39 -07:00
Teng Ma
96a5e4dd79 [Feature] Support loading weights from ckpt engine worker (#11755) 
Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
Co-authored-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-10-23 09:23:30 -07:00
Lianmin Zheng
9eefe2c0b7 Set CUDA_VISIBLE_DEVICES to achieve one GPU per process (#9170) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-10-17 17:30:06 -07:00
Lianmin Zheng
cd7e1bd591 Sync code and test CI; rename some env vars (#11686) 2025-10-15 18:37:03 -07:00
Liangsheng Yin
acc2327bbd Move deep gemm related arguments to sglang.srt.environ (#11547) 2025-10-14 00:34:35 +08:00
Lianmin Zheng
dda34c2f93 Fix mem fraction static for nightly tests (#11076) 2025-09-29 12:57:41 -07:00
Philip Kiely - Baseten
7f028b07c4 Fix formatting in long code blocks (#10528) 2025-09-16 12:02:05 -07:00
Feng Su
4c21b09074 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 1 (#9962) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
Signed-off-by: Huaixin Chang <changhuaixin@linux.alibaba.com>
Signed-off-by: Peng Wang <rocking@linux.alibaba.com>
 2025-09-15 02:08:02 +08:00
Shu Wang
3df05f4d6a [NVIDIA] [3/N] Nvfp4 Masked Gemm: Add flashinfer grouped_gemm_nt_masked (#9199) 2025-09-11 20:18:43 -07:00
Liangsheng Yin
6e95f5e5bd Simplify Router arguments passing and build it in docker image (#9964) 2025-09-05 12:13:55 +08:00
Huapeng Zhou
75ee00112d [Doc] Fix SGLang tool parser doc (#9886) 2025-09-04 21:52:53 +08:00
Lifu Huang
1fbfdebe6b [chore] fix dead links in doc (#9913) 2025-09-02 00:28:26 -07:00
Liangsheng Yin
f9afa7dceb Fix docs for clip max new tokens (#9082) 2025-08-11 13:15:21 -07:00
Lianmin Zheng
2449a0afe2 Refactor the docs (#9031) 2025-08-10 19:49:45 -07:00
Yineng Zhang
9020f7fc32 chore: bump v0.5.0rc0 (#8959) 2025-08-08 09:16:18 -07:00
Yineng Zhang
8cd344586e chore: bump v0.4.10.post2 (#8727) 2025-08-03 03:43:29 -07:00
Cheng Wan
6c88f6c8d9 [5/N] MoE Refactor: Update MoE parallelism arguments (#8658) 2025-08-01 01:20:03 -07:00
Ke Bao
33f0de337d chore: bump v0.4.10.post1 (#8652) 2025-08-01 12:07:30 +08:00
Faraz
4b04998d38 TRTLLM Gen MLA Decode Kernel Integration (same as #7938) (#8632) 
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
 2025-07-31 16:03:40 -07:00
Yineng Zhang
023288645b chore: bump v0.4.10 (#8608) 2025-07-31 20:50:17 +08:00
Yineng Zhang
6478831be9 chore: bump v0.4.9.post6 (#8517) 2025-07-29 02:30:07 -07:00
Yineng Zhang
45bc170b36 chore: bump v0.4.9.post5 (#8458) 2025-07-28 02:11:06 -07:00
Qiaolin Yu
484d0e021d doc: add bench_one_batch_server in the benchmark doc (#8441) 2025-07-27 23:07:54 -07:00
Yineng Zhang
2272c2a5b5 chore: bump v0.4.9.post4 (#8305) 2025-07-25 17:12:47 -07:00
Xiaoyu Zhang
9045cc1eb8 [torch.compile bug] avoid biased_grouped_topk_impl func repeatedly triggering torch.compile in forward pass (#8353) 2025-07-25 21:17:47 +08:00
Zaili Wang
15d2759174 [CPU] Add tutorial docs for SGL on CPU (#8000) 2025-07-25 00:03:16 -07:00
Yineng Zhang
01c000043c chore: bump v0.4.9.post3 (#8265) 2025-07-22 15:55:48 -07:00
Yineng Zhang
22bd857cb5 docs: update README (#7985) 2025-07-12 13:31:11 -07:00
Yineng Zhang
eb118d88c4 chore: bump v0.4.9.post2 (#7963) 2025-07-11 21:11:20 -07:00
Yineng Zhang
066f4ec91f chore: bump v0.4.9.post1 (#7882) 2025-07-09 00:28:17 -07:00
Yineng Zhang
ec5f9c6269 chore: bump v0.4.9 (#7802) 2025-07-05 17:40:29 -07:00
Yuchen Cheng
1e3e3add3d fix(docs): fix the broken link in docs/references/production_metrics.md (#7741) 
Signed-off-by: rudeigerc <rudeigerc@gmail.com>
 2025-07-03 23:46:07 -07:00
Yi Zhang
93b6785d78 add description for llama4 eagle3 (#7688) 2025-07-01 01:19:19 -07:00
ybyang
7349717e4b [doc] update lws doc for pd (#7318) 2025-07-01 10:39:04 +08:00
tarinkk
eb6c2c1663 Hybrid kv cache for LLaMA4 (#6563) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-06-27 18:58:55 -07:00
Yineng Zhang
69183f8808 chore: bump v0.4.8.post1 (#7559) 2025-06-26 02:21:12 -07:00
Yineng Zhang
7c3a12c000 chore: bump v0.4.8 (#7493) 2025-06-23 23:14:22 -07:00
Yineng Zhang
f9dc9dd28b chore: bump v0.4.7.post1 (#7248) 2025-06-16 15:20:29 -07:00
Lianmin Zheng
90bd3e32d6 Improve perf tuning docs (#7071) 2025-06-10 16:55:04 -07:00
Yineng Zhang
4f723edd3b chore: bump v0.4.7 (#7038) 2025-06-10 01:56:20 -07:00
Yueyang Pan
98c00a2df1 Fix torch profiler bugs for bench_offline_throughput.py (#6557) 2025-06-09 20:33:41 +08:00
HAI
b819381fec AITER backend extension and workload optimizations (#6838) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
 2025-06-05 23:00:18 -07:00
Baizhou Zhang
791b3bfabb [Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479) 2025-05-28 16:03:43 -07:00