|
13bdc7bf4a
|
[Feature][NPU]: add runtime support for AutoRound quantized models (#16699)
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
|
2026-03-07 18:03:55 +03:00 |
|
|
|
6910c1b281
|
[Feature][NPU]: add runtime support for GPTQ-quantized MoE models (#16364)
Co-authored-by: GuoYechang <52730608+GuoYechang@users.noreply.github.com>
Co-authored-by: root <root@localhost.localdomain>
|
2026-03-04 16:02:19 +03:00 |
|
|
Yilong Zhao
|
79b1d2bac6
|
[loader] support presharded fused mlp loading (#19519)
|
2026-02-27 20:37:24 -08:00 |
|
|
Zheng Li
|
fa5698d791
|
feat: [Qwen3.5] Support block-wise FP8 quantization and model adaptation (#18926)
|
2026-02-18 11:44:25 +08:00 |
|
|
strgrb
|
37c33cc0aa
|
fuse qkvbfg linear into one gemm and f_b g_b into batched gemm. (#17801)
|
2026-02-04 11:41:26 +08:00 |
|
|
|
7412ceb4eb
|
[Auto Sync] Update linear.py to assert shapes (20260130) (#17966)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2026-01-31 01:01:55 -08:00 |
|
|
|
e86476acfc
|
[NPU] support llama-3.2-11B-vision-instruct mode for NPU (#17492)
Co-authored-by: McZyWu <zhuoyun.wu.23@ucl.ac.uk>
Co-authored-by: chenyang08056032 <chenyang08056032@163.com>
Co-authored-by: Hexq0210 <893781835@qq.com>
|
2026-01-31 08:49:38 +08:00 |
|
|
Zheng Liandyizhang2077
|
0c5a81acb8
|
[BUGFIX] Fix dp size > 1 for qwen3 vl model (#17624)
Co-authored-by: yizhang2077 <1109276519@qq.com>
|
2026-01-30 20:44:25 +08:00 |
|
|
22dimensions
|
7b79326751
|
[NPU] support GPTQ quantization on npu (#15203)
Signed-off-by: 22dimensions <waitingwind@foxmail.com>
|
2026-01-29 15:48:18 +08:00 |
|
|
Sam Shleifer
|
0d49b13fdd
|
Fix circular import in quantization modules (#17372)
|
2026-01-21 15:47:09 +08:00 |
|
|
|
5af84c8af5
|
[AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392)
Co-authored-by: Dehua Tang <dehtang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
|
2026-01-14 01:44:40 -08:00 |
|
|
Lianmin Zheng
|
075c5a5789
|
Code clean up for fp8 quantization (#16982)
|
2026-01-13 12:38:39 -08:00 |
|
|
+6
|
160a06cab2
|
[Feature] Xiaomi MiMo-V2-Flash day0 support (#15207)
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
|
2025-12-19 11:40:07 +08:00 |
|
|
ZhengdQin
|
c05d3afb5d
|
[NPU] optimization for dsv3.2 (#14572)
|
2025-12-12 14:52:16 +08:00 |
|
|
|
21028b5507
|
[RL] support weight reload for low-bit rollout (#9650)
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2025-12-10 15:44:01 +08:00 |
|
|
fzyzcjy
|
64a11303ce
|
Fix update weight error for blackwell DeepGEMM (#13910)
|
2025-11-25 13:28:12 -08:00 |
|
|
Lianmin Zheng
|
c7d57d5bb3
|
Fix CI and style (#12658)
|
2025-11-05 15:08:15 -08:00 |
|
|
Nicolas Castet
|
2340798353
|
Register allgather/reducescatter buffers with symm memory (#12572)
|
2025-11-04 17:11:36 -08:00 |
|
|
Lianmin ZhengandNicolas Castet
|
7a21d8b276
|
Reduce the overhead of nccl symmetric memory (#12524)
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
|
2025-11-03 11:56:27 -08:00 |
|
|
|
39c237f02c
|
Add AWQ quantization support for NPU. (#10158)
Co-authored-by: Alisehen <814073252@qq.com>
Co-authored-by: Yaochen Han <48639761+Alisehen@users.noreply.github.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
|
2025-10-23 12:08:05 -07:00 |
|
|
jacky.cheng
|
99c92ff24b
|
[AMD] Support a new flag to disable quant on parallelLinear layer if required (#11811)
|
2025-10-22 19:16:15 -07:00 |
|
|
Yueyang Pan
|
8260574729
|
fix: fp8 quantization failure of qwen 2.5 VL 7B model (#10112)
Signed-off-by: PanJason <pyyjason@gmail.com>
|
2025-09-27 05:05:23 +00:00 |
|
|
|
fac07c9b08
|
Support LingV2 model (#10359)
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
Co-authored-by: guoyuhong <yuhong.gyh@antgroup.com>
|
2025-09-11 23:53:52 -07:00 |
|
|
Lianmin Zheng
|
4582931ac3
|
Revert "Revert the changes on NCCL symmetric memory" (#10238)
|
2025-09-09 12:11:49 -07:00 |
|
|
Lianmin ZhengandYineng Zhang
|
d352c29aa0
|
Revert the changes on NCCL symmetric memory (#10210)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-09 11:01:33 -07:00 |
|
|
Yineng Zhang
|
45b3a6a256
|
Revert "[ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization (#9712)" (#10176)
|
2025-09-08 11:28:15 -07:00 |
|
|
Pavani Majety
|
fcd72bd100
|
[ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization (#9712)
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
|
2025-08-29 17:13:52 -07:00 |
|
|
blzheng
|
ebbb75e917
|
[CPU] Fix TP padding issue on Phi-4 (#8289)
|
2025-08-17 16:25:26 -07:00 |
|
|
Xiaoyu Zhang
|
44e86480e8
|
fuse allreduce and residual_rmsnorm (#8731)
|
2025-08-11 13:50:53 -07:00 |
|
|
Trevor Morris
|
c0e84297c2
|
Use reduce scatter for DP (#8539)
|
2025-08-06 16:21:26 -07:00 |
|
|
Ying Sheng
|
c1d2061f97
|
Add initial support for gpt-oss (#8824)
|
2025-08-05 13:42:01 -07:00 |
|
|
Nicolas Castet
|
82e6c3a65a
|
Add support for NCCL symmetric memory for TP allreduces (#8238)
|
2025-08-01 23:30:55 +00:00 |
|
|
Cheng Wan
|
15ad6c9086
|
[1/N] MoE Refactor: refactor select_experts (#7966)
|
2025-07-19 00:51:15 -07:00 |
|
|
|
d918ab7985
|
Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs (#7302)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
|
2025-07-18 19:59:39 -07:00 |
|
|
Cheng Wan
|
49b8777460
|
Refactor: move all quantization-related code to srt/layer/quantization (#7989)
|
2025-07-17 00:47:07 -07:00 |
|
|
Xiaoyu Zhang
|
49a5915f53
|
[ready b200] fuse allreduce+add_rmsnorm in prepare_attention + mlp module (#7775)
|
2025-07-10 15:12:39 -07:00 |
|
|
|
766392c6bd
|
[feature]Ascend quantization support (#7791)
Co-authored-by: ichernob <ichernobnn@gmail.com>
Co-authored-by: liupeng <liupeng374@huawei.com>
|
2025-07-10 09:17:37 -07:00 |
|
|
Chunyuan WU
|
1dce6c480f
|
[CPU] support the case where num_attention_heads or intermediate_size is not divisible by the TP size (#6771)
|
2025-07-03 09:51:38 -07:00 |
|
|
Chunyuan WU
|
9fcc9a80e7
|
[CPU] refine CPU integration code (#7647)
|
2025-07-03 09:51:09 -07:00 |
|
|
Chunyuan WUandThien Tran
|
7eb47b0f3d
|
[CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641)
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2025-06-25 01:43:33 -07:00 |
|
|
Baizhou Zhang
|
2a5f0100e0
|
Fix GGuf and add back test_gguf.py (#7067)
|
2025-06-10 21:07:20 -07:00 |
|
|
|
8f783c1943
|
[Model Support] unsloth/Phi-4-mini bnb model (#4982)
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-04-16 19:58:20 -07:00 |
|
|
Trevor Morris
|
11d760d56a
|
FP4 weight loading and inference (2/2) (#3972)
|
2025-04-08 17:26:21 -07:00 |
|
|
Xiaoyu Zhang
|
04e3ff6975
|
Support compressed tensors fp8w8a8 (#4743)
|
2025-03-26 13:21:25 -07:00 |
|
|
|
f44db16c8e
|
[Feature] Integrate DeepEP into SGLang (#4232)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
|
2025-03-19 08:16:31 -07:00 |
|
|
|
588865f0e0
|
[Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274)
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-03-18 20:33:07 -07:00 |
|
|
HandH1998
|
0dd6cda288
|
Apply sgl w8a8 fp8 kernel (#3148)
|
2025-03-09 00:03:32 -08:00 |
|
|
|
ac2387279e
|
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
|
2025-03-03 00:12:04 -08:00 |
|
|
laixinandHandH1998
|
1a6e97577a
|
Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)
Co-authored-by: HandH1998 <1335248067@qq.com>
|
2025-02-24 05:43:35 -08:00 |
|
|
Liangsheng Yin
|
8616357a97
|
Fix deepseek awq v3 (#3450)
|
2025-02-12 22:09:52 +08:00 |
|