 cctry
|
b36afed4a7
|
Separate allocation logic from scheduler (#11313)
|
2025-10-10 17:38:54 -07:00 |
|
|
JinYan Suandhuangtingwei
|
9aa4502d11
|
feat(mooncake): support GB suffix for global_segment_size (#10745)
Signed-off-by: Jinyang Su <751080330@qq.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
|
2025-10-10 17:38:25 -07:00 |
|
|
Scott Lee
|
55b14656e6
|
Revert "Add metrics for speculative decoding (acceptance rate, average acceptance length)" (#11433)
|
2025-10-10 12:54:57 -07:00 |
|
|
Lianmin Zheng
|
b4408e6098
|
Revert "fix: fix video input for qwen3-vl" (#11437)
|
2025-10-10 12:44:40 -07:00 |
|
|
Cheng Wan
|
52fcbbb8bd
|
Revert "perf: optimize qwen-vl with symm mem allreduce" (#11436)
|
2025-10-10 12:30:05 -07:00 |
|
|
Teng Ma
|
9082a7d323
|
[HiCache] feat: add multi tenant with prefix tag (#9256)
|
2025-10-11 00:23:28 +08:00 |
|
|
Yuan Luoandluoyuan.luo
|
3b9d97f335
|
perf: optimize qwen-vl with symm mem allreduce (#11381)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-10-10 22:24:45 +08:00 |
|
|
Mick
|
a1a20b4c7c
|
fix: fix video input for qwen3-vl (#11361)
|
2025-10-10 04:35:35 -07:00 |
|
|
Yineng Zhang
|
4299aebdbb
|
chore: update pyproject (#11420)
|
2025-10-10 00:56:30 -07:00 |
|
|
Scott Lee
|
0babd48736
|
Add metrics for speculative decoding (acceptance rate, average acceptance length) (#11144)
|
2025-10-10 00:46:44 -07:00 |
|
|
Zaili Wang
|
f19613e6c3
|
Dedicated toml files for CPU/XPU (#10734)
|
2025-10-10 00:44:55 -07:00 |
|
|
ziruiliu
|
8df4945559
|
fix file and object naming scheme in HiCacheNixl to avoid data corruption (#10969)
Signed-off-by: Zirui Liu <ziliu@ddn.com>
|
2025-10-10 00:23:10 -07:00 |
|
|
|
ee3bd8a1c8
|
feat(hicache): Support passing prefix keys for l3 store. (#9045)
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-10-10 00:22:05 -07:00 |
|
|
Yuan Luoandluoyuan.luo
|
b5044fbf12
|
Replace pad with cat for better performance (#11388)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-10-10 12:03:17 +08:00 |
|
|
Glen Liu
|
9a7e7a6576
|
[Bug Fix] prevent lora adapter from being loaded into LoRAManager if it is already loaded (#11365)
|
2025-10-09 18:43:03 -07:00 |
|
|
Yingchun Lai
|
0fe87213bb
|
fix: fix gpu-proc affinity set incorrectly when pp_size > 1 (#11389)
|
2025-10-09 18:40:05 -07:00 |
|
|
Xinyuan Tong
|
1f106ee365
|
[grammar] Avoid server crash when grammar backend is None (#11401)
|
2025-10-09 18:38:10 -07:00 |
|
|
Lianmin Zheng
|
9b8ebb2798
|
move more files under srt/utils (#11285)
|
2025-10-09 16:46:15 -07:00 |
|
|
sglang-bot
|
758b887ad1
|
chore: bump SGLang version to 0.5.3.post1 (#11324)
|
2025-10-09 15:19:59 -07:00 |
|
|
Yineng Zhang
|
44cb060785
|
chore: upgrade flashinfer 0.4.0 (#11364)
|
2025-10-09 14:17:54 -07:00 |
|
|
Chang Su
|
b520958ec8
|
[router][grpc] Replace fake health check with correct ones (#11387)
|
2025-10-09 09:13:57 -07:00 |
|
|
shaharmor98
|
fa7e2c3049
|
fix bench_serving mishandling of internal states (#11376)
Signed-off-by: Shahar Mor <smor@nvidia.com>
|
2025-10-09 19:24:50 +08:00 |
|
|
shaharmor98
|
8f2cd177af
|
add code pp support for nixl (#11375)
Signed-off-by: Shahar Mor <smor@nvidia.com>
|
2025-10-09 19:24:32 +08:00 |
|
|
Trevor Morris
|
a4b424c632
|
[DeepSeek-V3.2] Include indexer kv cache when estimating kv cache size (#11309)
|
2025-10-08 23:59:46 -07:00 |
|
|
Simo Lin
|
368fd20622
|
[router][grpc] disable health check generation and increase timeout (#11353)
|
2025-10-08 19:23:08 -07:00 |
|
|
Sundara Raman Ramachandran
|
53bd00d975
|
[Generative Score API] Multi-Item scoring with custom attention mask. (#10979)
|
2025-10-08 18:47:32 -07:00 |
|
|
|
e22b13c569
|
[Auto Sync] Update scheduler.py (20251009) (#11350)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Junxiong Wang <junxiong@together.ai>
|
2025-10-08 17:39:04 -07:00 |
|
|
Chang Su
|
a65ca73911
|
[router][grpc] Cleanup debug logs in grpc_server and grpc_router (#11340)
|
2025-10-08 13:26:19 -07:00 |
|
|
Netanel Haber
|
d6837aea4d
|
model: Support Hybrid Mamba2 NemotronHForCausalLM (nvidia/NVIDIA-Nemotron-Nano-9B-v2) (#10909)
Signed-off-by: Netanel Haber <nhaber@nvidia.com>
|
2025-10-09 00:37:38 +08:00 |
|
|
Liangsheng YinandShangming Cai
|
c882b5ae75
|
[CI] improve disaggregation CI. (#11264)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2025-10-08 21:40:56 +08:00 |
|
|
|
e3bb7f5ae6
|
benchmark: enhance configurable multimodal benchmarking in bench_serving (#9812)
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-10-08 01:31:36 -07:00 |
|
|
Lifu Huang
|
92473e2e34
|
Support LoRA in bench_serving oai interface (#11318)
|
2025-10-08 01:28:58 -07:00 |
|
|
JinYan Su
|
6c0bb32711
|
fix(decode): adjust ServerArgs import to explicit module path (#11007)
|
2025-10-08 01:27:50 -07:00 |
|
|
Lifu HuangandHieu Pham
|
edefab0c64
|
[2/2] Support MHA prefill with FlashAttention 4. (#10937)
Co-authored-by: Hieu Pham <hyhieu@gmail.com>
|
2025-10-08 00:54:20 -07:00 |
|
|
Cheng Wan
|
97cd38e58d
|
Skip weight loading in deepgemm compilation (#11312)
|
2025-10-07 21:52:46 -07:00 |
|
|
Cheng Wan
|
3c06b673af
|
[8/N] MoE Refactor: deprecate EPMoE (#11211)
|
2025-10-07 21:51:41 -07:00 |
|
|
Adarsh Shirawalmath
|
7c3f07dbcb
|
[Feature] Add /tokenize and /detokenize OpenAI compatible endpoints (#9545)
|
2025-10-08 12:38:48 +08:00 |
|
|
Liangsheng Yin
|
4b4dc132fa
|
Rename ngram_utils -> ngram_info (#11316)
|
2025-10-08 11:49:46 +08:00 |
|
|
YAMYandXinyuan Tong
|
5a9170d993
|
Optimize copy_kv_cache for spec decoding (#11126)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-10-08 10:43:30 +08:00 |
|
|
DarkSharpness
|
832c84fba9
|
[Chore] Update xgrammar 0.1.24 -> 0.1.25 (#10710)
|
2025-10-07 18:22:28 -07:00 |
|
|
Mick
|
64d1505c0a
|
ci: unify the model launch method of nightly ci (#11230)
|
2025-10-07 18:13:14 -07:00 |
|
|
cctry
|
f3764c26a3
|
Clean match_prefix and prepare_for_extend for mem cache V2 (#11200)
|
2025-10-07 17:54:18 -07:00 |
|
|
Chang Su
|
7ba3de0e92
|
[oai serving chat] Add argument --sampling-defaults and fix ChatCompletionRequest defaults (#11304)
|
2025-10-08 00:36:05 +00:00 |
|
|
Chang Su
|
f094e0a490
|
[router][grpc] Fix request_id extraction when n > 1 (#11311)
|
2025-10-07 19:27:56 -04:00 |
|
|
Bowen Bao
|
cd4b39a900
|
[quantization] Properly ignore quantization for layers excluded in quant_config (#11205)
|
2025-10-07 14:06:05 -07:00 |
|
|
Xinyuan Tong
|
e3c7f09146
|
Update tool parser and related documentation (#11223)
|
2025-10-07 11:03:40 -07:00 |
|
|
Chang Su
|
6f1e03a456
|
[router][grpc] Fix sampling_params.stop_strs is None (#11306)
|
2025-10-07 10:57:38 -07:00 |
|
|
Simo Lin
|
2fcd56eaf6
|
[router] add get server info and get model info in grpc server (#11303)
|
2025-10-07 08:36:52 -07:00 |
|
|
Liangsheng Yin
|
3ddd7dc9f8
|
Introduce future indices (#11301)
|
2025-10-07 22:24:02 +08:00 |
|
|
Liangsheng Yin
|
501dfa6b42
|
Remove sampling info events and overlap thread file (#11300)
|
2025-10-07 21:34:25 +08:00 |
|