Simo Lin
|
9ad02b799d
|
[Perf] Optimize radix tree for cache-aware load balancin (#14758)
|
2025-12-09 15:35:29 -08:00 |
|
Lianmin Zheng
|
18bd8e8d6d
|
Improve CI by trying a warmup before unit tests (#14669)
|
2025-12-09 15:17:59 -08:00 |
|
Lianmin Zheng
|
036e64dafa
|
move multi-item scoring functions in tokenizer manager into a separate file (#14740)
|
2025-12-09 14:47:06 -08:00 |
|
Douglas Yang
|
7c6fb3aa2d
|
fix: making rate limit a warning instead of error (#14753)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-09 13:41:09 -08:00 |
|
Douglas Yang
|
8b0b6a45c8
|
fix: checking if tokenizer is in cache before downloading from HF (#14698)
|
2025-12-09 13:08:07 -08:00 |
|
b8zhong
|
55504df2f7
|
Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379)
|
2025-12-09 12:05:56 -08:00 |
|
Simo Lin
|
73df7a4e8d
|
[SMG] perf: optimize tokenizer for reduced CPU and memory overhead (#14752)
|
2025-12-09 11:41:13 -08:00 |
|
Simo Lin
|
8b98bb768c
|
[model-gateway] optimize core modules (#14751)
|
2025-12-09 10:54:07 -08:00 |
|
Yuhao Yang
|
15bc8cbd74
|
fix rope parameter initialization error caused by transformers v5.0 update (#14745)
|
2025-12-09 10:51:26 -08:00 |
|
yudian0504
|
9496f12d00
|
[Model] Add PaddleOCR-VL Model Support (#12953)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-12-09 10:16:02 -08:00 |
|
Lianmin Zheng
|
ab0048793c
|
Revert "[Feat] Add received_time in serving_base" (#14743)
|
2025-12-09 06:52:27 -08:00 |
|
blahblah
|
6ec77680df
|
[diffusion] feat: support comparing batch perf (#14738)
Co-authored-by: shuxiguo <shuxiguo@meituan.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
|
2025-12-09 22:48:52 +08:00 |
|
Alison Shao
|
13680e5542
|
[Test] Skip STANDALONE speculative decoding tests for different hidden sizes (#14733)
|
2025-12-09 22:21:35 +09:00 |
|
yrk111222
|
98c430e114
|
fix: prevent HugginqFace access when SGLANG_USE_MODELSCOPE is enabled (#12039)
|
2025-12-09 21:02:05 +08:00 |
|
zhanghaotong
|
fe7f91ef82
|
[Feat] Add received_time in serving_base (#13432)
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
|
2025-12-09 22:00:24 +09:00 |
|
kun-llfl
|
cef5ba65b1
|
[Bugfix] Fix environ error in scheduler_runtime_checker_mixin.py (#14461)
Signed-off-by: Kun(llfl) <i@imux.top>
|
2025-12-09 20:33:41 +08:00 |
|
Xiaoyu Zhang
|
53d170883a
|
Add fuse_marlin_moe test to ci and add new ep test (#14686)
|
2025-12-09 20:17:38 +08:00 |
|
Rain Jiang
|
f0e948a0f1
|
fix the deepep 8 gpu unit test (#14601)
|
2025-12-09 01:40:09 -08:00 |
|
Alison Shao
|
9a426fc5ef
|
[CI] Move mistral large 3 basic to nightly (#14622)
|
2025-12-09 00:28:44 -08:00 |
|
Yineng Zhang
|
66772aa2b4
|
chore: add code owners for deepseek_v2.py (#14714)
|
2025-12-08 23:24:06 -08:00 |
|
Yineng Zhang
|
b626334475
|
fix: make override DeepseekV2Model work (#14707)
|
2025-12-08 23:11:18 -08:00 |
|
b8zhong
|
0f8bd55f3e
|
[CI] Fix Llama 3.1 8B FP4 CI (#14699)
|
2025-12-08 22:27:15 -08:00 |
|
fzyzcjy
|
da3dc497b0
|
Fix dp-aware incompatible with completions and chat completions APIs (#14647)
|
2025-12-09 13:24:53 +08:00 |
|
fzyzcjy
|
817daba062
|
Tiny extract select_worker_min_load (#14648)
|
2025-12-08 21:22:10 -08:00 |
|
Simo Lin
|
af60cad05d
|
[ci][smg] fix docker release ci and add it to pr test (#14683)
|
2025-12-08 20:56:38 -08:00 |
|
Lianmin Zheng
|
ce4e836be5
|
Add per-request decode tp size (#14678)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2025-12-08 20:24:31 -08:00 |
|
Yineng Zhang
|
0e0b0c0566
|
Revert "[Bug] fix not desired disable fused share experts caused by r… (#14676)
|
2025-12-08 20:06:52 -08:00 |
|
Alison Shao
|
e6f0ddda44
|
[CI] Migrate Eagle 1-GPU tests to test/registered/ (#14529)
|
2025-12-09 12:56:36 +09:00 |
|
fzyzcjy
|
af20657cd4
|
Tiny support sgl-router http response status code metrics (#14689)
|
2025-12-08 19:54:35 -08:00 |
|
Alison Shao
|
08da4c2618
|
[Bugfix] Fix KeyError for Mistral-Large-3 rope_scaling config (#14627)
|
2025-12-08 19:16:00 -08:00 |
|
Binyao Jiang
|
ef3f8c97e1
|
Add ffmpeg into sglang docker - required by transformers multimodal V… (#14679)
|
2025-12-08 18:00:23 -08:00 |
|
Baizhou Zhang
|
e5201bda34
|
[CI] Unblock gb200 cutedsl test (#14469)
|
2025-12-08 17:58:25 -08:00 |
|
Even Zhou
|
60d36e7be7
|
[NPU] chore: bump basic software version to 8.3.rc2 (#14614)
|
2025-12-09 09:14:27 +08:00 |
|
Jimmy
|
6f657070ef
|
[SMG]feat: implement TokenGuardBody for managing token return (#14653)
|
2025-12-08 16:44:46 -08:00 |
|
kk
|
c106b54b57
|
Aiter fp8 kv cache (#13147)
|
2025-12-08 16:39:53 -08:00 |
|
fzyzcjy
|
119fd956fb
|
Tiny support printing requests in bench_serving for observability (#14652)
|
2025-12-09 08:27:58 +08:00 |
|
PiteXChen
|
eac5b66485
|
[RadixTree] Optimize the Time Complexity of Node Retrieval Operation from O(n*m) to O(n) (#13334)
Signed-off-by: CLFutureX <chenyongqyl@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-12-08 15:55:09 -08:00 |
|
Francis
|
07404d7689
|
[HiCache] fix condition check when use decode offload (#14489)
|
2025-12-08 15:52:40 -08:00 |
|
Alison Shao
|
93043f7b13
|
[CI] Increase max-parallel to 15 for high priority PRs (#14675)
|
2025-12-08 15:40:37 -08:00 |
|
Simo Lin
|
edde5e5d40
|
[model-gateway] add OTEL integration to grpc router (#14671)
|
2025-12-08 14:45:05 -08:00 |
|
Binyao Jiang
|
6abb8051e8
|
Bump up diffusers to latest official release version (#14670)
|
2025-12-08 13:41:01 -08:00 |
|
fzyzcjy
|
2e3946d889
|
Fix cache-aware router should pick min load instead of min tenant size (#14650)
|
2025-12-08 13:33:34 -08:00 |
|
b8zhong
|
32f8b6064e
|
improve default glm mtp setting (#14457)
Signed-off-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-08 13:27:13 -08:00 |
|
Yuhao Yang
|
b9bef31a15
|
fix: use .get() when accessing strict mem-check env variable (#14657)
|
2025-12-08 13:25:42 -08:00 |
|
Simo Lin
|
8550822d6b
|
[model-gateway] Optimize memory usage in HTTP router (#14667)
|
2025-12-08 13:10:31 -08:00 |
|
Mick
|
8810152e88
|
vlm: Use fa3 as the default backend for qwen3 vl (#14634)
|
2025-12-08 12:56:20 -08:00 |
|
Simo Lin
|
7bf16c6339
|
[model-gateway] fix WASM arbitrary file read security vol (#14664)
|
2025-12-08 12:11:16 -08:00 |
|
Simo Lin
|
39f9a9c2a5
|
[model-gateway] reduce cpu overhead in grpc router (#14663)
|
2025-12-08 11:54:56 -08:00 |
|
Simo Lin
|
d69ecc19b8
|
[model-gateway] reducing cpu overhead in various of places (#14658)
|
2025-12-08 09:44:40 -08:00 |
|
yctseng0211
|
763888b5a8
|
[AMD] change fused rms quant interface for aiter upgrade (#14497)
|
2025-12-08 09:09:23 -08:00 |
|