chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
6,562 Commits 1 Branch 0 Tags
2104d20eba2b3badcc51c3d8fbcec48f45e045e5
Commit Graph

328 Commits

Author SHA1 Message Date
Atream
2104d20eba Temporarily fix missing routed_scaling_factor for CompressedTensorsWNA16MoEMethod (#12738) 2025-11-06 12:01:03 +08:00
YAMY
f235498eca DeepSeek-V3.2: Add Adaptive MHA Attention Pathway for Short-Sequence Prefill (#11892) 2025-11-05 19:33:26 -08:00
Nicolas Castet
2340798353 Register allgather/reducescatter buffers with symm memory (#12572) 2025-11-04 17:11:36 -08:00
Trevor Morris
dbcf85b7f0 Add --speculative-moe-runner-backend server arg (#10183) 2025-11-04 00:20:56 -08:00
Lianmin Zheng
7a21d8b276 Reduce the overhead of nccl symmetric memory (#12524) 
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
 2025-11-03 11:56:27 -08:00
Atream
14d8064803 fix: Fix KTransformers hybrid inference with int8 quantization and format (#12536) 2025-11-03 04:59:39 -08:00
Ke Bao
a4bf5c6ad2 Support Kimi Linear (#12469) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-10-31 14:03:35 -07:00
Even Zhou
ce6b17c0f9 [Feature] Support DeepSeek MTP on NPU (#11897) 
Co-authored-by: liupeng374 <liupeng374@huawei.com>
 2025-10-30 21:56:56 +08:00
JensenFire
78c43d8847 [Feature] Initial eagle3 support for Deepseek-like models (#12319) 2025-10-30 17:26:09 +08:00
Trevor Morris
9ff9fa7f95 Fuse wk and weight_proj in Indexer for DeepSeekV3.2-FP4 (#12094) 2025-10-29 18:42:56 -07:00
Minglei Zhu
e39628fd07 [2/2] Deepseek deterministic: support deepseek v3 deterministic inference on 8 x H200 (#12095) 2025-10-29 11:49:04 -07:00
Gao016
0297773a2f a tiny fix for support deepseek bf16 weights (#12313) 
Co-authored-by: gaochang <gaochang@U-19PX2WQ1-0350.local>
 2025-10-28 20:46:44 -07:00
Cheng Wan
83087247d1 [hotfix] missing w13_weight_fp8 and w2_weight_fp8 in UE8M0 requantization (#12259) 2025-10-28 19:10:38 -07:00
weiliang
88596739a4 Support running FP4 Deepseek on SM120. (#11708) 2025-10-27 17:37:49 -07:00
Cheng Wan
649949807f [10/N] MoE Refactor: reorganize deepgemm runner in DeepEPMoE (#12054) 2025-10-24 19:16:17 -07:00
Minglei Zhu
f4b78d137c [1/2] deepseek deterministic: support deterministic inference for deepseek arch models on a single GPU (#12000) 2025-10-24 15:17:28 -07:00
Yongfei Xu
4793ec7d1a Opt MHA chunked prefix: merge prefix and extend kv cache to run mha once (#10953) 2025-10-23 20:58:10 -07:00
Mick
770529a731 model: support deepseek-ocr (#11891) 
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Shi Shuai <126407087+shuaills@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-10-24 03:15:17 +08:00
ErvinXie
39c237f02c Add AWQ quantization support for NPU. (#10158) 
Co-authored-by: Alisehen <814073252@qq.com>
Co-authored-by: Yaochen Han <48639761+Alisehen@users.noreply.github.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
 2025-10-23 12:08:05 -07:00
Liangsheng Yin
9d61205dac [lint] improve ruff check (#11922) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-10-22 11:32:50 +08:00
Atream
7e6191c098 init support for KTransformers Heterogeneous Computing (#11487) 
Co-authored-by: Jianwei Dong <1913953267@qq.com>
 2025-10-21 00:17:02 -07:00
Cheng Wan
bfc3b3f786 [9/N] MoE Refactor: cleanup dispatcher interfaces (#11847) 2025-10-20 10:11:46 -07:00
Liu-congo
be0058bc05 [BugFix] replace the input_to_float8 used in dsv2 (#11612) 
Signed-off-by: Liu-congo <1502632128@qq.com>
 2025-10-19 19:34:13 -05:00
fzyzcjy
12eb02e982 Change bf16 to fp8 for some gemms in attention for DeepSeek ckpt v2 (#11805) 2025-10-19 16:15:13 +08:00
fzyzcjy
ce399e154c Make single-batch overlap compatible with NextN (#11804) 2025-10-19 16:10:44 +08:00
Cheng Wan
5b214b50b6 [Refactor] move deep_gemm_wrapper out of quantization (#11784) 2025-10-17 18:57:54 -07:00
fzyzcjy
33e9bbec35 Make single-batch overlap compatible with offloading (#11614) 2025-10-18 08:45:54 +08:00
fzyzcjy
dcb8f090ad Super tiny fix CI (#11788) 2025-10-17 17:41:58 -07:00
fzyzcjy
8af8491298 Support casting bf16 NextN moe to fp8 (#11613) 2025-10-18 08:02:15 +08:00
fzyzcjy
505329cab0 Support shared experts overlap in cutlass moe (#11611) 2025-10-18 07:59:40 +08:00
Chang Su
627974405d [Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685) 2025-10-17 16:49:46 -07:00
Even Zhou
3cceaa381a [Bugfix] Fix Qwen3/DSV3/DSV3.2 model support (#11510) 2025-10-16 15:14:09 +08:00
Xun Sun
a40229f6f8 [1/N] Introduce Mooncake Backend and Mooncake EP to Support Elastic EP (#10423) 
Co-authored-by: Hank Han <hanhan7630@outlook.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-10-14 19:40:54 -07:00
Liangsheng Yin
516738b096 Depreate global_server_args_dict (#11528) 2025-10-13 19:34:43 +08:00
Cheng Wan
1bdd010291 Revert "Deprecate global_server_args_dict" (#11520) 2025-10-12 17:40:40 -07:00
Liangsheng Yin
1083e7e3df Deprecate global_server_args_dict (#11331) 2025-10-13 01:20:47 +08:00
Liu-congo
c80a96dae9 [BugFix] test_mla_fp8.py fails on Cublas 12.9 (#11360) 
Signed-off-by: Liu-congo <1502632128@qq.com>
 2025-10-10 21:14:24 -07:00
fzyzcjy
efbc687c28 Support DeepSeek V3.2 Exp (#11061) 
Co-authored-by: Stefan He <11166516+hebiao064@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <56809903+fridge003@users.noreply.github.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
Co-authored-by: ZhengdQin <46387172+zhengdqin@users.noreply.github.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-06 00:24:15 -07:00
fzyzcjy
b65db0287b Tiny cleanup deepseek_v2.py (#11163) 2025-10-02 21:54:52 +08:00
fzyzcjy
5e786cca3a Support single batch overlap (#10422) 2025-10-02 18:04:36 +08:00
fzyzcjy
0b9dfba787 Support dispatch low latency (#10263) 
Co-authored-by: Kaixi Hou <4001424+kaixih@users.noreply.github.com>
 2025-10-02 18:02:19 +08:00
fzyzcjy
f35def8652 Fuse quantize and rope in trtllm_mla MTP (#10779) 2025-10-02 17:59:37 +08:00
fzyzcjy
44b1fbe258 Fix DeepSeek chunked prefill memory issue (#11149) 2025-10-01 23:56:59 -07:00
Even Zhou
d27a6f7092 [Feature] Add MLAProcess for DeepSeek MLA on NPU (#10130) 2025-09-22 17:17:48 -07:00
Yineng Zhang
f67d1f45bc [Auto Sync] Update deepseek_v2.py (20250922) (#10717) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Granado <mgranado@together.ai>
 2025-09-21 17:43:50 -07:00
Yineng Zhang
7c876de7f5 fix: remove awq_dequantize deps (#10686) 2025-09-20 01:47:01 -07:00
Yineng Zhang
b17e67df36 [Auto Sync] Update deepseek_v2.py (20250920) (#10683) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 23:43:31 -07:00
Shu Wang
124097fc5b enable prefix cache with dp (#10459) 2025-09-16 18:26:58 -07:00
cicirori
a2f7218a2e support using fa4 on deepseek on blackwell (#9928) 2025-09-16 16:16:06 -07:00
fzyzcjy
059c13de5c Fix trtllm_moe wrong correction bias (#10440) 2025-09-15 01:02:05 -07:00