 Ke Bao
|
7b7e357f61
|
Separate swa and local attention chunk cache eviction (#15820)
|
2025-12-26 09:34:22 +08:00 |
|
|
Lianmin Zheng
|
8087ef126f
|
Clean up the __init__ of TokenizerManager and DetokenizerManager (#15796)
|
2025-12-25 10:30:27 -08:00 |
|
|
Huaixin Chang
|
0c39730b18
|
DP: support piggyback server load report (#11469)
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
|
2025-12-25 11:35:05 +08:00 |
|
|
satyamk7054andSatyam Kumar
|
38dd4fbb66
|
Add overlap scheduling for embeddings code path (#14032)
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
|
2025-12-24 18:24:18 -08:00 |
|
|
fzyzcjy
|
8196998a03
|
Tiny add num retracted tokens metric (#15653)
|
2025-12-24 22:15:23 +08:00 |
|
|
fzyzcjy
|
fd4a558e71
|
Add metrics for having prefill and decode in different ranks (#15752)
|
2025-12-24 21:35:35 +08:00 |
|
|
ratish
|
5f3a47d8a7
|
[model-gateway]: add gRPC router embeddings endpoint implementation (#15273)
|
2025-12-23 09:12:22 -08:00 |
|
|
Liangsheng Yin
|
705287b2e5
|
Tiny add more information in retract logging. (#15694)
|
2025-12-24 00:43:36 +08:00 |
|
|
Tianyu Guo
|
fa2966983a
|
Support PP for zmq_to_scheduler (#15312)
|
2025-12-23 17:07:55 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
3c882db3ad
|
Adjust wrong mtp meaning introduce by mimo (#15632)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-23 02:06:46 +08:00 |
|
|
fzyzcjy
|
454a2544f2
|
Support soft watchdog for tokenizer/detokenizer/dp-controller processes (#15607)
|
2025-12-22 16:59:16 +08:00 |
|
|
|
bed301a5ac
|
[Feature] Enable return routed experts (#12162)
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-12-21 15:16:43 +08:00 |
|
|
Liangsheng Yin
|
933cef16cc
|
Tiny fix mimo model conflicts with main (#15483)
|
2025-12-19 23:20:59 +08:00 |
|
|
Lianmin Zheng
|
f228b662a7
|
Update readme (#15425)
|
2025-12-18 23:06:00 -08:00 |
|
|
+6
|
160a06cab2
|
[Feature] Xiaomi MiMo-V2-Flash day0 support (#15207)
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
|
2025-12-19 11:40:07 +08:00 |
|
|
Zehuan Li
|
f6c9db4bc4
|
[DLLM] Fix dLLM regression (#15371)
|
2025-12-19 10:55:46 +08:00 |
|
|
fzyzcjy
|
88a405cc10
|
Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401)
|
2025-12-18 22:24:23 +08:00 |
|
|
fzyzcjy
|
c5f4e20f2f
|
Support GPU execution time breakdown by forward mode metrics (#15396)
|
2025-12-18 22:18:17 +08:00 |
|
|
Lianmin Zheng
|
d1f0063262
|
Clean up __init__ function of the scheduler and event loop for PD (#15298)
|
2025-12-18 01:35:14 -08:00 |
|
|
fzyzcjy
|
b12b40dea4
|
Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189)
|
2025-12-17 21:19:42 +08:00 |
|
|
Lianmin Zheng
|
9d64a7b24f
|
Minor style fixes to the scheduler.py (#15218)
|
2025-12-16 17:09:44 -08:00 |
|
|
amysaq2023
|
ccc8f3b266
|
support non disturbing remote instance weight loader v2 (#14997)
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
|
2025-12-16 14:39:56 -08:00 |
|
|
Hanming Lu
|
e61dabf5e4
|
[Qwen3-next] support mamba radix cache for overlap scheduler (#14792)
|
2025-12-14 18:54:16 -08:00 |
|
|
|
9acb21ae27
|
feat: support EPD disaggregation (#12263)
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: Nicholas <45984215+liusy58@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
|
2025-12-14 22:30:08 +08:00 |
|
|
fzyzcjy
|
168a31eb00
|
Support prefill max requests limitation (#14993)
|
2025-12-14 10:52:10 +08:00 |
|
|
liupeng374
|
3134d2b2a7
|
[scheduler] enhance scheduler in dp_attention mixed case with spec (#14201)
|
2025-12-14 02:52:26 +08:00 |
|
|
Liangsheng Yin
|
ed52d01b0b
|
Fix spec info's filter when reqs are finished right after prefill (#14742)
|
2025-12-14 00:32:54 +08:00 |
|
|
Liangsheng Yin
|
01e3b3f3a3
|
Fix decode OOM caused by retraction (#14939)
|
2025-12-13 12:59:17 +08:00 |
|
|
fzyzcjy
|
313f59ad80
|
Add soft watchdogs to debug soft hangs (#15023)
|
2025-12-13 10:41:35 +08:00 |
|
|
fzyzcjy
|
487cf81a68
|
Tiny extract SchedulerWatchdog (#15021)
|
2025-12-13 10:40:32 +08:00 |
|
|
Chenxi Li
|
9b9d21312a
|
Feature/Fix multi lora scheduler blocking issue and evict LoRA None lastly (#14795)
|
2025-12-12 17:13:05 -08:00 |
|
|
Yineng Zhangandfzyzcjy
|
4b7b5af36a
|
Revert several PRs (#14958)
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
|
2025-12-12 11:25:12 -08:00 |
|
|
Kevin LiandShangming Cai
|
8fa8d9d7e8
|
[PD] Add decode PP event loop for PD disaggregation (#14945)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2025-12-12 20:15:00 +08:00 |
|
|
|
c01b2ee094
|
[PP] Refactor PP to async mode (#11852)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: bluecoffee8 <jasperli2002@gmail.com>
Co-authored-by: zhangxiaolei123456 <zhangxiaolei.666@bytedance.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
|
2025-12-12 12:54:16 +08:00 |
|
|
Xinyuan Tong
|
9975acf50f
|
[refactor] Update reasoning parameter to require_reasoning (#14922)
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-12-11 16:55:59 -08:00 |
|
|
amysaq2023
|
70758d457e
|
support non-disturbing remote-instance-weight-loader (#13125)
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
|
2025-12-11 16:45:32 -08:00 |
|
|
ShawnY112358and赵晨阳
|
f077436831
|
[fix] Fix issues for in-flight weight updates (#14064)
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
|
2025-12-09 18:12:19 -08:00 |
|
|
b8zhong
|
55504df2f7
|
Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379)
|
2025-12-09 12:05:56 -08:00 |
|
|
Muqi LiandXinyuan Tong
|
06836ad02a
|
[Reasoning + Structured Output] make reasoning compatible with structured output (#12551)
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-12-08 01:28:39 -08:00 |
|
|
roikoren755
|
2ce121a1c3
|
Enable RadixCache for Mamba2 models (#13584)
|
2025-12-05 18:23:58 +08:00 |
|
|
|
64092c8b55
|
[Auto Sync] Rename is_hybrid to is_hybrid_swa (#14252)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
|
2025-12-01 23:24:24 -08:00 |
|
|
liupeng374andEven Zhou
|
2e8f54e61e
|
[spec-overlap] bugfix for pd disaggregation and npu (#14088)
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
|
2025-12-01 22:58:20 +08:00 |
|
|
fzyzcjy
|
340c613ab5
|
Support numactl bind for CPU and memory before process starts (#14156)
|
2025-11-30 17:00:33 +08:00 |
|
|
fzyzcjy
|
45cf575852
|
Fix overlap scheduler not take effect when outputing logprobs (#14096)
|
2025-11-28 18:15:56 +08:00 |
|
|
|
ab9a46d462
|
Support configuring the request limit per receiving poll (#14076)
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
Co-authored-by: Feng Su <225349073+sufeng-buaa@users.noreply.github.com>
|
2025-11-28 16:14:21 +08:00 |
|
|
Yixin DongandLiangsheng Yin
|
6350042696
|
feat: Naive support Spec V2 + Constrained Decoding (#13425)
Signed-off-by: Ubospica <ubospica@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-11-27 20:31:46 +08:00 |
|
|
fzyzcjy
|
25758647b1
|
Support sanity checking weight consistency especially for RL (#13854)
|
2025-11-27 20:25:12 +08:00 |
|
|
Liangsheng Yin
|
eff6a07c8f
|
Fix get_load API (#13991)
|
2025-11-26 22:07:51 +08:00 |
|
|
vipwangerxiaoandPeng Wang
|
15729dbc8e
|
Use dynamically maintained num_waiting_tokens in get_load() (#13203)
Signed-off-by: Peng Wang <peng_wang@linux.alibaba.com>
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
|
2025-11-26 20:34:45 +08:00 |
|
|
Zehuan LiandTiwei Bie
|
21b0582d4b
|
[feature] Initial block diffusion language model support (#12588)
Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
|
2025-11-26 17:57:54 +08:00 |
|