Chang Su
|
83804bc626
|
[router][grpc] Restructure modules and code clean up (#12598)
|
2025-11-03 23:59:27 -08:00 |
|
Zhao Chen
|
d5fa019c36
|
feat: limit peak memory usage when computing logprobs (#6318)
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
|
2025-11-03 23:53:20 -08:00 |
|
Ben Barsdell
|
fef3a6b63b
|
Restore torch defaults between sgl-kernel tests (#11131)
|
2025-11-03 23:51:23 -08:00 |
|
Junrong Lin
|
173e0f704f
|
Enable memory saver for hybrid model (#11974)
|
2025-11-04 14:55:26 +08:00 |
|
Lianmin Zheng
|
f600866a44
|
Improve the metrics for PD (#12580)
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: cctry <shiyang@x.ai>
|
2025-11-03 22:10:57 -08:00 |
|
ishandhanani
|
93be7e863e
|
fix: respect --ignore-eos in PD case for benchmarking (#12597)
|
2025-11-03 21:44:14 -08:00 |
|
fzyzcjy
|
60b0754cc9
|
Tiny fix ExpertDistributionReq error (#11760)
|
2025-11-04 13:39:25 +08:00 |
|
Zhao Chen
|
0b24af4d79
|
test: support return logprobs in bench_offline_throughput test (#12462)
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-11-03 21:38:48 -08:00 |
|
Jonah Bernard
|
a209fb05c1
|
[Qwen3 VL] Add LoRA support for Qwen 3 VL (#12165)
|
2025-11-03 20:32:54 -08:00 |
|
Hanming Lu
|
48d6bea1ea
|
[GDN/SWA] mamba and swa radix cache edge case fix (#12111)
Co-authored-by: yizhang2077 <1109276519@qq.com>
|
2025-11-04 11:03:37 +08:00 |
|
Teng Ma
|
1689c0e35f
|
[Doc] fix miss index for production request trace (#12547)
|
2025-11-03 17:57:09 -08:00 |
|
fzyzcjy
|
193fbb0bce
|
Super tiny add UT for copy_to_gpu_no_ce (#12270)
|
2025-11-04 09:40:51 +08:00 |
|
akhilg-nv
|
e607850fcf
|
Enable mixed type LayerNorm kernel for NSA indexer (#12044)
|
2025-11-03 16:50:41 -08:00 |
|
Baizhou Zhang
|
15efbcb4e7
|
[chore] Fix update_kernel_whl_index script for multiple cuda version (#12519)
|
2025-11-03 16:34:14 -08:00 |
|
Lianmin Zheng
|
243c064df2
|
Remove the dependency of nccl.h in symmetric memory (#12571)
|
2025-11-03 16:11:00 -08:00 |
|
Chang Su
|
0b41a293fa
|
[router][grpc] Consolidate error messages build in error.rs (#12301)
|
2025-11-03 14:41:45 -08:00 |
|
b8zhong
|
d31d48b341
|
update usage of trtllm_fp8_per_tensor_scale_moe (#12569)
|
2025-11-03 14:25:32 -08:00 |
|
fzyzcjy
|
8834260739
|
Super tiny dump server info such as args in bench for post analysis (#12550)
|
2025-11-03 14:24:08 -08:00 |
|
fzyzcjy
|
fd7a72d62d
|
Super tiny allow profile activities in bench_serving (#12549)
|
2025-11-03 14:23:18 -08:00 |
|
Yi Zhang
|
21a8fa16ea
|
tiny optimize for bench serving (#12553)
|
2025-11-03 14:13:18 -08:00 |
|
Lianmin Zheng
|
7a21d8b276
|
Reduce the overhead of nccl symmetric memory (#12524)
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
|
2025-11-03 11:56:27 -08:00 |
|
R0CKSTAR
|
d36639eec7
|
[ROCm] Update Mooncake to v0.3.7.post1 and add -DUSE_HIP=ON to rocm.Dockerfile (#12560)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
|
2025-11-03 11:33:30 -08:00 |
|
Jonah Bernard
|
6ef23b9833
|
[Test] Add parameters to SRTRunner (#12227)
|
2025-11-03 11:20:56 -08:00 |
|
fzyzcjy
|
385599cb04
|
Fix error when calling quantization (#12548)
|
2025-11-03 10:17:43 -08:00 |
|
Yueyang Pan
|
952fbe47cb
|
fix: fix the bug which leads qwen2_5_vl to crash with mixed_chunk (#11330)
Signed-off-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
|
2025-11-03 09:26:03 -08:00 |
|
Liangsheng Yin
|
edb2569356
|
[hot-fix] Fix broken CI (#12564)
|
2025-11-04 00:03:25 +08:00 |
|
Liangsheng Yin
|
3529c061bb
|
[spec v2] Fix output repetition by speculative sampling error (#12561)
|
2025-11-03 23:00:17 +08:00 |
|
harrisonlimh
|
ffb32a8548
|
Conditionally recapture cuda graph after model weight update from disk (#12060)
|
2025-11-03 05:51:27 -08:00 |
|
Atream
|
14d8064803
|
fix: Fix KTransformers hybrid inference with int8 quantization and format (#12536)
|
2025-11-03 04:59:39 -08:00 |
|
Shangming Cai
|
ab8b83f71d
|
chore: upgrade mooncake 0.3.7.post1 (#12541)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2025-11-03 00:34:32 -08:00 |
|
yinghui
|
de0b10cf5c
|
fix: move dummy format loader check before quantization checks (#12532)
|
2025-11-02 23:41:30 -08:00 |
|
Baizhou Zhang
|
6e29446e45
|
[hotfix] Remove flashinfer-jit-cache from pyproject (#12530)
|
2025-11-02 22:11:05 -08:00 |
|
Yineng Zhang
|
0c3543d7d5
|
chore: upgrade flashinfer 0.5.0 (#12523)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-11-02 20:54:12 -08:00 |
|
Kangyan-Zhou
|
6a3b9fd00f
|
Update setup_github_runner.md
|
2025-11-02 20:44:09 -08:00 |
|
Haian Huang(深度眸)
|
65f1d065c5
|
[Bug] Fix Intern-S1 model accuracy and support /generate interface with input_ids (#12367)
|
2025-11-02 20:22:33 -08:00 |
|
Johnsonms
|
9434a0e50f
|
[Refact] Remove hardcoded KV cache dimension in MLATokenToKVPool (#12502)
|
2025-11-02 19:49:53 -08:00 |
|
Lianmin Zheng
|
20315697f4
|
move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521)
|
2025-11-02 13:15:05 -08:00 |
|
fzyzcjy
|
c9db79117f
|
Super tiny fix naming in bench serving scripts (#12515)
|
2025-11-02 12:43:10 -08:00 |
|
bingps
|
15ed27d7d4
|
[Fix] concat_mla_absorb_q_kernel fails for long inputs (#12453)
|
2025-11-02 11:52:06 -08:00 |
|
Hanming Lu
|
66fb9b1307
|
[ServerArgs] allow --mamba-ssm-dtype extend (#12481)
|
2025-11-02 11:50:04 -08:00 |
|
Yuan Luo
|
819fc59123
|
Add prefix for torch symm mem (#12506)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-11-02 11:23:05 -08:00 |
|
kousakawang
|
7efd8b3d1f
|
[FEAT] Shared mem pool based cuda ipc for multi-modal data transport (#11917)
Co-authored-by: kousakawang <wanghanpei@bytedance.com>
Co-authored-by: Yuan Luo <4908075+yuan-luo@users.noreply.github.com>
|
2025-11-02 16:46:37 +08:00 |
|
sglang-bot
|
a920b9dace
|
chore: bump sgl-kernel version to 0.3.16.post5 (#12511)
|
2025-11-02 14:08:11 +08:00 |
|
Ho-Ren (Jack) Chuang
|
76196b3cbf
|
feat: Add FP4 (E2M1) KV Cache Support with Quantization Utilities for MLA (#10078)
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
Co-authored-by: Yichen Wang <yichen.wang@bytedance.com>
|
2025-11-01 22:24:58 -07:00 |
|
Xiaoyu Zhang
|
95191ebdca
|
Migrate weak_ref_tensor to sgl-kernel (#12505)
|
2025-11-02 10:55:39 +08:00 |
|
Baizhou Zhang
|
9a512cf95b
|
[CI] Move some Lora/Deterministic CI tests to nightly (#12507)
|
2025-11-01 19:54:22 -07:00 |
|
Binyao Jiang
|
3451fc3280
|
[Feature] Qwen3-Next & FLA: Support MTP topk>1; Up to 6% faster (#11133)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-11-01 19:47:56 -07:00 |
|
Zhihao Lyu
|
c550ab9125
|
[Ascend] Add Ascend NPU support for sglang.check_env & rework proposal (#11052)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
|
2025-11-01 19:26:45 -07:00 |
|
Even Zhou
|
086f0b79fc
|
[CI][NPU] remove pypi mirror site that hangs ci dependency installation (#12499)
|
2025-11-01 19:25:53 -07:00 |
|
Xun Sun
|
0afd68321b
|
Update Mooncake EP's a2a interface (#12391)
|
2025-11-01 18:48:47 -07:00 |
|