-
8601300beb
fix: fix the error where the content is None when reasoning and tool … (#5838)
mlmz
2025-04-28 23:36:08 +08:00
-
6fa6f38ed3
Feat: add support for thinking mode via chat_template_kwargs.enable_t… (#5551)
mlmz
2025-04-28 22:07:45 +08:00
-
693723d1f7
Revert "Tiny refactor DefaultModelLoader.Source" (#5825)
Lianmin Zheng
2025-04-28 01:18:57 -07:00
-
966eb90865
[Docs] Replace lists with tables for cleanup and readability in server_arguments (#5276)
Michael Yao
2025-04-28 15:36:10 +08:00
-
644ed409d1
Tiny refactor DefaultModelLoader.Source (#5482)
fzyzcjy
2025-04-28 15:35:51 +08:00
-
3029889cb4
Turn on overlap scheduler for multimodal models (#5771)
Lianmin Zheng
2025-04-27 23:45:09 -07:00
-
ef15dcda26
Add a doc to fix sgl-kernel build link error in py39 with ccache (#5809)
Xiaoyu Zhang
2025-04-28 12:34:27 +08:00
-
ad4df30741
Dockerfile.dev pip scikit_build_core (#5807)
Xiaoyu Zhang
2025-04-28 12:14:20 +08:00
-
41ac0c6d48
chore: upgrade sgl-kernel 0.1.0 (#5690)
Yineng Zhang
2025-04-27 21:00:50 -07:00
-
84810da4ae
Add Cutlass MLA attention backend (#5390)
Trevor Morris
2025-04-27 20:58:53 -07:00
-
40d9b8acce
Improve overlap scheduling (#5788)
Liangsheng Yin
2025-04-28 11:19:16 +08:00
-
f0365820e8
[Misc] add structure logging, write to file and log tracing for SGL Router
Simo Lin
2025-04-27 16:54:10 -07:00
-
86317c09e9
[Docs] update grafana setup guide in production metrics (#5643)
Huapeng Zhou
2025-04-28 06:36:33 +08:00
-
daed453e84
[CI] Improve github summary & enable fa3 for more models (#5796)
Lianmin Zheng
2025-04-27 15:29:46 -07:00
-
ded04b2e0a
Update nightly-test.yml (#5797)
Lianmin Zheng
2025-04-27 15:27:24 -07:00
-
84022c0e56
Release v0.4.6 (#5795)
Baizhou Zhang
2025-04-27 14:07:05 -07:00
-
f9fb33efc3
Add 8-GPU Test for Deepseek-V3 (#5691)
Baizhou Zhang
2025-04-27 12:46:12 -07:00
-
a38f6932cc
[CI] Fix test case (#5790)
Lianmin Zheng
2025-04-27 08:55:35 -07:00
-
beb65c7433
[PD]Reduce kv transfer threads (#5791)
Liangsheng Yin
2025-04-27 23:03:30 +08:00
-
621e96bf9b
[CI] Fix ci tests (#5769)
Lianmin Zheng
2025-04-27 07:18:10 -07:00
-
35ca04d2fa
[CI] fix port conflicts (#5789)
Lianmin Zheng
2025-04-27 05:17:44 -07:00
-
3c4e0ee64d
[CI] Tune threshold (#5787)
Lianmin Zheng
2025-04-27 04:10:22 -07:00
-
9c088829ee
Revert "Use device_id in dist init to reduce NCCL communicator warmup & creation overhead" (#5786)
Lianmin Zheng
2025-04-27 04:03:02 -07:00
-
005aad32ad
Revert "[fix] fix bench_one_batch_server" (#5785)
Lianmin Zheng
2025-04-27 03:48:33 -07:00
-
4d23ba08f5
Simplify FA3 tests (#5779)
Lianmin Zheng
2025-04-27 01:30:17 -07:00
-
6e313c1b8b
Revert "Revert "fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512"" (#5777)
Lianmin Zheng
2025-04-27 01:04:15 -07:00
-
a45a4b239d
Split local attention test from fa3 test (#5774)
Baizhou Zhang
2025-04-27 01:03:31 -07:00
-
981a2619d5
Fix eagle test case (#5776)
Lianmin Zheng
2025-04-27 01:00:54 -07:00
-
8ba313304d
Revert "fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512" (#5772)
Lianmin Zheng
2025-04-26 23:26:08 -07:00
-
021020632a
add switch to disable open api doc (#3744)
zhanweidu
2025-04-27 14:18:47 +08:00
-
7e944246c3
Add memory_saver check (#4986)
Kebe
2025-04-27 11:20:05 +08:00
-
a086a11305
Use sgl-kernel sgl_per_token_group_quant_int8 (#4971)
lambert0312
2025-04-27 11:19:42 +08:00
-
bdbe5f816b
update llguidance to 0.7.11; adds StructTag (#4870)
Michał Moskal
2025-04-26 20:13:57 -07:00
-
9ad28f639e
fix(srt): check if sample_indices is not None before usage. (#5633)
aoshen524
2025-04-26 22:51:01 -04:00
-
d7b1ce65a5
Handle JSONDecodeError while processing request data (#5599)
yan97ao
2025-04-27 10:50:50 +08:00
-
f55933e1cc
[misc] more decode step log for batch_one_batch (#5565)
JieXin Liang
2025-04-27 10:50:28 +08:00
-
408ba02218
Add Llama 4 to FA3 test (#5509)
Stefan He
2025-04-26 19:49:31 -07:00
-
094891c01a
fix: Use
is not None instead of != None for None checks. (#5687)
vzed
2025-04-26 22:26:57 -04:00
-
a21ef36352
support for the DeepSeek model by enabling streaming response parsing (#5592)
Frankey_8080
2025-04-27 09:59:31 +08:00
-
3c4dc38a9a
[fix] fix bench_one_batch_server (#5607)
JieXin Liang
2025-04-27 09:49:45 +08:00
-
d8fbc7c096
[feature] support for roberta embedding models (#5730)
DavidBao
2025-04-27 09:47:06 +08:00
-
c5e1026f47
Update amd docker image to
sglang:v0.4.5.post3-rocm630. (#5697)
saienduri
2025-04-26 18:46:57 -07:00
-
799c4bb502
Fuse MLA set kv cache kernel (#5748)
Ke Bao
2025-04-27 09:42:22 +08:00
-
02723e1b0d
CI: rewrite test_vision_chunked_prefill to speedup (#5682)
Mick
2025-04-27 10:33:13 +09:00
-
df2cf583ce
we fix the non existent access of
decrypted_config_file (#5685)
vzed
2025-04-26 21:32:37 -04:00
-
133ded039a
perf: update H20 fused_moe_triton kernel config to get higher throughput during prefilling (#5716)
saltyfish66
2025-04-27 09:15:07 +08:00
-
f87a6ab359
Resolves the
404 Not Found error when running compile_deep_gemm.py in multi-node setups (#5720)
Yuhong Guo
2025-04-27 09:13:13 +08:00
-
eebfdb9459
[fix] fix potential bumpy throughtput with deepgemm (#5722)
JieXin Liang
2025-04-27 09:12:48 +08:00
-
dfb322642f
Use device_id in dist init to reduce NCCL communicator warmup & creation overhead (#5728)
Wenxuan Tan
2025-04-26 20:11:09 -05:00
-
63c13a2c73
fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512 (#5733)
Kyungmin Lee
2025-04-27 10:10:23 +09:00
-
4d1e52abea
Add an assertion to enhance the robustness of the operator (#5736)
liwenju0
2025-04-27 09:09:12 +08:00
-
155890e4d1
[Minor] fix documentations (#5756)
Lianmin Zheng
2025-04-26 17:48:43 -07:00
-
1f963d7f64
Bugfix for minicpmo vision test (#5760)
Yi Zhang
2025-04-26 23:18:02 +08:00
-
04d0123fd9
[Fix]: support deepseek-vl2-tiny model (#5552)
ZXN
2025-04-26 17:52:53 +08:00
-
feda9b11b3
fix: fix one more bug from merging mm_inputs (#5718)
Mick
2025-04-26 09:28:33 +09:00
-
c3948ba67e
Reorder loop in shared expert weight loading (#5719)
Ke Bao
2025-04-26 08:27:42 +08:00
-
269c457e05
[Docs] Update runtime/engine/readme.md (#5737)
Michael Yao
2025-04-26 07:39:29 +08:00
-
18ce468d56
update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740)
Xiaoyu Zhang
2025-04-26 07:24:59 +08:00
-
21514ff5bd
Disable flaky eagle tests (#5753)
Lianmin Zheng
2025-04-25 15:54:39 -07:00
-
5641a09458
Revert "[Model] Support
ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct)" (#5754)
Lianmin Zheng
2025-04-25 15:50:28 -07:00
-
3dd3538c18
Pin torch audio to 2.6.0 (#5750)
Lianmin Zheng
2025-04-25 15:06:28 -07:00
-
93c6fb12c7
Fix: deepseek forward absorb (#5723)
michael-amd
2025-04-25 13:48:55 -07:00
-
11e27d0926
[PD]: Support Muti Prefill in one node (#5704)
IAN
2025-04-26 00:30:47 +08:00
-
50eda8398e
[PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738)
shangmingc
2025-04-25 18:15:01 +08:00
-
c55550cbf0
[PD] Better logs (#5715)
Liangsheng Yin
2025-04-25 17:25:45 +08:00
-
43fb95c2fa
[Model] Support
ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct) (#5078)
Brayden Zhong
2025-04-25 03:24:09 -04:00
-
7d9679b74d
Add MMMU benchmark results (#4491)
Ravi Theja
2025-04-25 12:53:53 +05:30
-
b5be56944b
[Doc] Fix a link to Weilin Zhao (#5706)
Michael Yao
2025-04-25 02:02:27 +08:00
-
d2b8d0b8d8
Add example to use sgl engine with fastapi (#5648)
Ravi Theja
2025-04-24 21:27:05 +05:30
-
a14654dd68
Fix weight loading bug for Deepseek v3+nextn (#5684)
Baizhou Zhang
2025-04-24 06:29:56 -07:00
-
5d93a950ee
[BugFix] Fix combination of MTP and
--n-share-experts-fusionwith R1 (#5707)
Yuhong Guo
2025-04-24 21:13:51 +08:00
-
c998d04b46
vlm: enable radix cache for qwen-vl models (#5349)
Mick
2025-04-24 12:35:05 +09:00
-
7d0edf3cae
chore: bump sgl-kernel 0.1.0 (#5688)
Yineng Zhang
2025-04-23 14:23:59 -07:00
-
ce4ecba477
fix: only compile ApplyTokenBitmaskInplace cu124+ (#5686)
Yineng Zhang
2025-04-23 14:17:42 -07:00
-
b1f6d89b5f
fix: update truss bench_serving (#5683)
Yineng Zhang
2025-04-23 13:28:35 -07:00
-
7c99103f4c
[Doc] Fix two 404 links caused by sglang typo (#5667)
Michael Yao
2025-04-23 23:21:55 +08:00
-
de071366cd
tune the threshold of gemma-2-27b-it in test_nightly_gsm8k_eval.py (#5677)
Lianmin Zheng
2025-04-23 05:31:17 -07:00
-
e0673969b9
[PD] Add support for dp attention with mooncake (#5530)
shangmingc
2025-04-23 17:20:27 +08:00
-
127ff8982e
fix torchvision::nms not exist (#5671)
Yineng Zhang
2025-04-23 02:17:21 -07:00
-
8777a1d24b
fix gemma3 unit test (#5670)
Yineng Zhang
2025-04-23 02:14:01 -07:00
-
711efe7814
Integrating PD disaggregation with DP attention and DeepEP (#5435)
Cheng Wan
2025-04-23 01:46:01 -07:00
-
fbb5f229d4
fix awq_dequantize import (#5669)
Yineng Zhang
2025-04-23 01:36:26 -07:00
-
15fabcc07f
fix sgl-kernel unit tests (#5666)
Yineng Zhang
2025-04-23 01:18:30 -07:00
-
e62c49557d
[1/2] Add FP8 Blockscale MoE CUTLASS kernel for Blackwell (#5281)
Elfie Guo
2025-04-22 22:28:20 -07:00
-
71d1785f2d
Remove unnecessary
torch.full in DeepSeek (#5601)
fzyzcjy
2025-04-23 12:24:29 +08:00
-
3f87f83116
Fuse q_a_proj and kv_a_proj (#5619)
Baizhou Zhang
2025-04-22 20:35:08 -07:00
-
ce5412b62e
Turn on DeepGemm By Default and Update Doc (#5628)
Baizhou Zhang
2025-04-22 16:10:08 -07:00
-
7282ab741a
fix: update bench_speculative (#5649)
Yineng Zhang
2025-04-22 16:08:15 -07:00
-
b0feda090c
Revert "Support aiter RMSNorm in AMD" (#5646)
HAI
2025-04-22 15:20:24 -07:00
-
6b6e748775
Remove q concat in FA3 backend for DeepSeek decode (#5638)
Ke Bao
2025-04-23 02:43:12 +08:00
-
917324862e
[fix] reduce dp capture bs (#5634)
JieXin Liang
2025-04-23 02:08:45 +08:00
-
2ed96c7a8a
fix flashmla bug (#5272)
lukec
2025-04-23 01:36:23 +08:00
-
2aa3f5e2d0
[feature] Add H20 fp8_w8a8 FusedMoE config for --n-share-experts-fusion=16 (#5641)
saltyfish66
2025-04-23 00:33:13 +08:00
-
76d17c7ecb
Fix shared experts fusion error without quantization (#5632)
lambert0312
2025-04-23 00:22:26 +08:00
-
70d040f904
[NFC] Remove duplicate
compressed-tensors (#5640)
Connector Switch
2025-04-23 00:10:25 +08:00
-
4418f599a5
Fix FA3 DeepSeek prefill performance regression (#5624)
JieXin Liang
2025-04-22 16:41:41 +08:00
-
04f2abcb34
fix: gemma 3 not use softcap (#5622)
Yineng Zhang
2025-04-22 01:16:08 -07:00
-
506be6b892
[fix] fix compile_deep_gemm missing kv_b_proj (#5620)
JieXin Liang
2025-04-22 15:06:36 +08:00
-
2343d8df7d
[fix] force use deepgemm in compile_deep_gemm (#5618)
JieXin Liang
2025-04-22 12:36:02 +08:00
-
92bb64bc86
[Doc] Fix a 404 link to llama-405b (#5615)
Michael Yao
2025-04-22 11:39:37 +08:00