 Baizhou Zhang
|
75e6a7cde1
|
Support radix cache for Lora feature (#7216)
|
2025-08-11 10:14:11 -07:00 |
|
|
Lianmin Zheng
|
2449a0afe2
|
Refactor the docs (#9031)
|
2025-08-10 19:49:45 -07:00 |
|
|
Lianmin Zheng
|
706bd69cc5
|
Clean up server_args.py to have a dedicated function for model specific adjustments (#8983)
|
2025-08-08 19:56:50 -07:00 |
|
|
Lianmin Zheng
|
a947154286
|
Revert "Support Multi Process Tokenizer Manager" (#8960)
|
2025-08-08 02:28:27 -07:00 |
|
|
pansicheng
|
e2fd2b9c7e
|
Simple prefetch policy (#8692)
|
2025-08-08 02:09:28 -07:00 |
|
|
|
7490e3f67d
|
Support Multi Process Tokenizer Manager (#6555)
Signed-off-by: ybyang <ybyang7@iflytek.com>
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: lw9527 <952799980@qq.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
|
2025-08-08 01:45:50 -07:00 |
|
|
Cheng Wan
|
1d24db8348
|
Expert Parallelism for GPT-OSS (#8944)
|
2025-08-08 00:46:42 -07:00 |
|
|
Xiaoyu Zhang
|
0d1e27a0c5
|
Better optimization log for gpt-oss model (#8953)
|
2025-08-08 00:11:48 -07:00 |
|
|
fzyzcjy
|
774b47f3f1
|
Reduce scheduler recv requests overhead (#8947)
|
2025-08-08 00:10:05 -07:00 |
|
|
Xiaoyu Zhang
|
76915d68a8
|
Fix enable flashinfer mxfp4 moe bf16 check (#8950)
|
2025-08-07 22:52:09 -07:00 |
|
|
Xiaoyu Zhang
|
47824c1488
|
[Perf] Auto enable best flashinfer mxfp4 kernel in b200 (#8898)
|
2025-08-07 01:08:41 -07:00 |
|
|
Xinyuan Tong
|
c36a6693f3
|
Disable gemma3 for SWA due to CUDA illegal memory access error (#8895)
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-08-07 00:44:44 -07:00 |
|
|
PGFLMG
|
b7cd743038
|
[Feat] QWen-1M context support[2/2]: Update block sparse attention backend (#5949)
|
2025-08-06 23:49:36 -07:00 |
|
|
Lifu Huang
|
6210e2c4f0
|
Support GPU pinning for LoRA (#8697)
|
2025-08-06 19:39:45 -07:00 |
|
|
eigenandaveryhuang
|
6ad6c8c9e6
|
feat: openai oss attention sink support with trtllm-gen backend #8825 (#8834)
Co-authored-by: averyhuang <averyh@nvidia.com>
|
2025-08-06 19:18:27 -07:00 |
|
|
Xiaoyu Zhang
|
4373df5525
|
add flashinfer mxfp4 (#8847)
|
2025-08-06 16:23:41 -07:00 |
|
|
|
92cc32d9fc
|
Support v1/responses and use harmony in serving_chat (#8837)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-08-06 16:20:34 -07:00 |
|
|
|
168033d5fb
|
Support mxfp4 for GPT-OSS (#8843)
Co-authored-by: Co-author fzyzcjy <ch271828n@outlook.com>
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
Co-authored-by: zhuofan1123 <zhuofanl@nvidia.com>
Co-authored-by: liz-badada <jinyanc@nvidia.com>
Co-authored-by: xutizhou <xutingz@nvidia.com>
Co-authored-by: linhu-nv <linhu@nvidia.com>
|
2025-08-06 00:05:25 -07:00 |
|
|
HouseWest
|
ca47e24f5d
|
[Feature] improve TBO: two chunk overlap (#8144)
|
2025-08-05 21:11:01 -07:00 |
|
|
Ke Bao
|
8128e08d36
|
Turn off hybrid cache by default (#8839)
|
2025-08-06 09:53:45 +08:00 |
|
|
Ying Sheng
|
c1d2061f97
|
Add initial support for gpt-oss (#8824)
|
2025-08-05 13:42:01 -07:00 |
|
|
eigenandBaizhou Zhang
|
40e3b2beeb
|
feat: add trtllm-gen mha from direct call (#8782)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-08-05 03:28:39 -07:00 |
|
|
|
d4bf5a8524
|
Support OCP MXFP4 quantization on AMD GPUs (#8255)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
|
2025-08-04 18:14:52 -07:00 |
|
|
Lifu Huang
|
7cb20754fa
|
[Fix] Fix several issues preventing gemma3n LoRA support. (#8776)
|
2025-08-04 17:11:46 -07:00 |
|
|
azhurkevichandCheng Wan
|
915140fd18
|
[NVIDIA] Add Low Latency NVFP4 decode kernels from Flashinfer (#8552)
Co-authored-by: Cheng Wan <cwan@x.ai>
|
2025-08-04 03:10:02 -07:00 |
|
|
Yingchun Lai
|
ed6f7597b3
|
Fix the missing 'lof' choice of --schedule-policy server args (#7114)
|
2025-08-03 12:29:42 -07:00 |
|
|
Guanhua Wang
|
f7b2853ff8
|
[feat] support minimum token load balance in dp attention (#7379)
|
2025-08-03 00:46:47 -07:00 |
|
|
Lifu Huang
|
8675bdf246
|
Support limiting max loaded loras in CPU. (#8650)
|
2025-08-03 00:02:23 -07:00 |
|
|
Lianmin Zheng
|
e314b084c5
|
[FIX] Fix the nightly CI by disabling swa mem pool for gemma2 (#8693)
|
2025-08-02 18:43:14 -07:00 |
|
|
Nicolas Castet
|
82e6c3a65a
|
Add support for NCCL symmetric memory for TP allreduces (#8238)
|
2025-08-01 23:30:55 +00:00 |
|
|
Cheng Wan
|
6c88f6c8d9
|
[5/N] MoE Refactor: Update MoE parallelism arguments (#8658)
|
2025-08-01 01:20:03 -07:00 |
|
|
Zhiqiang Xie
|
dd7ca00601
|
Interface change for kvcache io to support page first layout (#8318)
|
2025-08-01 11:37:49 +08:00 |
|
|
Kaixi Houandkushanam
|
aa4c66b564
|
[NVIDIA] Enable Flashinfer MoE blockscale fp8 backend for TP MoE (#8450)
Co-authored-by: kushanam <42385577+kushanam@users.noreply.github.com>
|
2025-07-31 19:56:34 -07:00 |
|
|
Faraz
|
4b04998d38
|
TRTLLM Gen MLA Decode Kernel Integration (same as #7938) (#8632)
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
|
2025-07-31 16:03:40 -07:00 |
|
|
Trevor Morris
|
b7170cc820
|
[bugfix] Fix flashinfer cutlass EP moe after MoE refactor (#8630)
|
2025-07-31 13:57:08 -07:00 |
|
|
|
2cd2e27f80
|
SGLang HiCache NIXL Connector (#8488)
Signed-off-by: Vishwanath Venkatesan <vvenkatesan@nvidia.com>
Co-authored-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-07-31 13:09:42 -07:00 |
|
|
Brayden Zhong
|
4acf690206
|
[Optimization][Perf] Disable the GC during CUDA graph capture to speed up by up to 3x (#8577)
|
2025-07-31 11:31:21 -07:00 |
|
|
Cheng Wan
|
7a1f7fc504
|
[Feature] Hybrid EP and TP (#8590)
|
2025-07-31 02:53:25 -07:00 |
|
|
|
51c38163c1
|
model: support Step3V (#8583)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-07-31 02:41:00 -07:00 |
|
|
|
d904959233
|
Support l3 cache (mooncake store) for hiradix cache (#7211)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
Co-authored-by: zuoyuan <zhangzuo21@mails.tsinghua.edu.cn>
Co-authored-by: @wangyueneng.wyn <wangyueneng.wyn@antgroup.com>
Co-authored-by: JinYan Su <jinyansu792@gmail.com>
|
2025-07-30 23:15:51 -07:00 |
|
|
pansichengandZhiqiang Xie
|
299803343d
|
Add hf3fs support for hicache storage (based on #7704) (#7280)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-07-30 17:42:41 -07:00 |
|
|
Lianmin Zheng
|
a4c3b121d8
|
Split the scheduler into multiple mixin classes to reduce the file size (#8483)
|
2025-07-29 12:46:50 -07:00 |
|
|
harrisonlimh
|
747dd45077
|
feat: throttle requests at scheduler based on --max_queued_requests (#7565)
|
2025-07-28 22:32:33 +08:00 |
|
|
|
6d6a8bc278
|
GLM-4.5 Model Support (#8224)
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-07-27 22:54:07 -07:00 |
|
|
|
2810338401
|
[feat] Support different attention backends for prefill and decode (#6338)
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-07-28 11:42:29 +08:00 |
|
|
Kaixi Hou
|
85486b6f6f
|
[NVIDIA] Add Flashinfer MoE blockscale fp8 backend (#8036)
|
2025-07-27 00:34:41 -07:00 |
|
|
fzyzcjy
|
da0c026084
|
Tiny assert EPLB is used together with expert parallel (#8381)
|
2025-07-26 03:20:39 -07:00 |
|
|
Trevor Morris
|
58c468f404
|
Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333)
|
2025-07-25 16:40:23 -07:00 |
|
|
Lianmin Zheng
|
ed2e313eb6
|
Clean up server_args, triton cache manager (#8332)
|
2025-07-25 14:14:51 -07:00 |
|
|
Chang Su
|
f8260f2539
|
[Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357)
|
2025-07-25 12:03:16 -07:00 |
|