 Ke Bao
|
c02e313914
|
Fix block wise fp8 torch compile (#3232)
|
2025-01-31 19:56:02 +08:00 |
|
|
Byron Hsu
|
734daedd8f
|
[fix] Clamp logprob with dtype min to prevent -inf (#3224)
|
2025-01-31 17:04:04 +08:00 |
|
|
Mick
|
9f635ea50d
|
[Fix] Address remaining issues of supporting MiniCPMV (#2977)
|
2025-01-28 00:22:13 -08:00 |
|
|
Byron Hsuandzhyncs
|
988d0a4bfc
|
[kernel] Use sgl_kernel rope (#3169)
Co-authored-by: zhyncs <me@zhyncs.com>
|
2025-01-28 14:33:11 +08:00 |
|
|
Yineng Zhang
|
2f79f58873
|
feat: use sgl-kernel 0.0.3 in sglang (#3179)
|
2025-01-27 21:39:52 +08:00 |
|
|
Lianmin Zheng
|
53cef81587
|
Improve weight loading and code style (#3174)
|
2025-01-27 03:00:41 -08:00 |
|
|
yigex
|
351a72d40b
|
add dsv3 mi300 triton config for block scale (#3146)
|
2025-01-27 17:25:53 +08:00 |
|
|
Lianmin Zheng
|
52c03f16b9
|
Add activation parameters to fused_moe (#3170)
|
2025-01-27 00:23:37 -08:00 |
|
|
Lianmin Zheng
|
1dda8c5e4c
|
Return more infos for computing average acceptance length (#3152)
|
2025-01-26 04:51:54 -08:00 |
|
|
Ke Wen
|
862bcff833
|
Support loading of larger models with on-the-fly quantization (#3061)
|
2025-01-22 21:33:17 -08:00 |
|
|
Lianmin Zheng
|
8b84e69f25
|
Fix tp token sync for dp attention (#3062)
|
2025-01-22 18:51:40 -08:00 |
|
|
Lianmin Zheng
|
022614d26e
|
Add some flags to allow sync token ids across TP ranks (#3060)
|
2025-01-22 15:05:51 -08:00 |
|
|
nstream-ai-devx
|
0d2148efaa
|
fix rotary_embedding rope_scaling for phi (#3055)
|
2025-01-23 02:15:32 +08:00 |
|
|
Lianmin Zheng
|
3d8f1c9bcf
|
Use int64 as indices for set_kv_buffer (#3039)
|
2025-01-21 19:46:09 -08:00 |
|
|
Lianmin Zheng
|
89cd923581
|
Roll back to use vllm custom allreduce (#3006)
|
2025-01-20 04:03:15 -08:00 |
|
|
Lianmin Zheng
|
dc1881326f
|
Fix perf regression on small batch sizes (#3008)
|
2025-01-20 03:39:49 -08:00 |
|
|
Hongpeng Guo
|
583697cd71
|
[Enhancement] Custom Logit Processor Improvement (#2998)
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
|
2025-01-20 02:00:35 -08:00 |
|
|
Yineng Zhang
|
44a9669770
|
keep rotary_embedding only (#2997)
|
2025-01-20 13:21:36 +08:00 |
|
|
Hongpeng Guo
|
e403d23757
|
[Feature] Add sampler custom logits processor (#2396)
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
|
2025-01-19 14:46:53 -08:00 |
|
|
Yineng Zhang
|
2c05f81f15
|
fix custom op version compatibility (#2988)
|
2025-01-20 04:21:29 +08:00 |
|
|
Yineng Zhang
|
6ada05d0ed
|
feat: check for is_cuda for sgl_kernel import (#2984)
|
2025-01-19 23:33:04 +08:00 |
|
|
Yineng Zhang
|
5a176c92df
|
fix deepseek v2 with cpu device (#2975)
|
2025-01-19 21:33:27 +08:00 |
|
|
Lianmin ZhengandSangBin Cho
|
23196d5254
|
Simplify logits processor (#2974)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
|
2025-01-18 23:03:49 -08:00 |
|
|
|
3d93f84a00
|
[Feature] Support minicpmv v2.6 (#2785)
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
|
2025-01-18 14:14:19 -08:00 |
|
|
Yineng Zhang
|
2add697d7a
|
feat: remove vllm get_rope (#2964)
|
2025-01-18 19:38:01 +08:00 |
|
|
Ke Bao
|
656dcc1a99
|
Remove fp8 monkey patch (#2960)
|
2025-01-18 15:00:29 +08:00 |
|
|
Yineng Zhang
|
033c715b46
|
cleanup models dependencies 1/n (#2948)
|
2025-01-17 23:46:48 +08:00 |
|
|
Yineng ZhangandZhangyi
|
5dc54f1a62
|
feat: remove vllm distributed (#2907)
Co-authored-by: Zhangyi <1109276519@qq.com>
|
2025-01-17 22:31:51 +08:00 |
|
|
Chunyuan WU
|
63051738a9
|
Enable CPU device on SGLang (#2806)
|
2025-01-16 21:22:53 -08:00 |
|
|
Lianmin Zhenganddhou-xai
|
8b6ce52e92
|
Support multi-node DP attention (#2925)
Co-authored-by: dhou-xai <dhou@x.ai>
|
2025-01-16 11:15:00 -08:00 |
|
|
Yineng Zhang
|
bf8d07a6f9
|
feat: patch linear base (#2915)
|
2025-01-16 18:00:03 +08:00 |
|
|
Lianmin Zheng
|
f65c13b559
|
Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902)
|
2025-01-15 04:54:14 -08:00 |
|
|
Ke Bao
|
cc0485bef2
|
Support w8a8 int8 quantization config (#2881)
|
2025-01-14 17:07:49 +08:00 |
|
|
Ke Bao
|
c19d84829c
|
Adjust flashinfer workspace size for Qwen2 models (#2879)
|
2025-01-14 13:34:22 +08:00 |
|
|
Lianmin Zheng
|
6249e4a19e
|
Revert "Integration of TurboMind AWQ" (#2866)
|
2025-01-13 04:44:39 -08:00 |
|
|
Ke Bao
|
f3516c2894
|
Fix quant kernel accuracy issue (#2865)
|
2025-01-13 20:32:17 +08:00 |
|
|
bjmsongandroot
|
17de02f98d
|
Integration of TurboMind AWQ (#2828)
Co-authored-by: root <bjmsong@126.com>
|
2025-01-13 20:14:16 +08:00 |
|
|
kkandLin, Soga <soga.lin@amd.com>
|
42f3909963
|
Unify sglang coding style (#2856)
Co-authored-by: Lin, Soga <soga.lin@amd.com>
|
2025-01-13 02:12:44 -08:00 |
|
|
Lianmin ZhengandSangBin Cho
|
72c7776355
|
Fix linear.py and improve weight loading (#2851)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
|
2025-01-13 01:39:14 -08:00 |
|
|
|
e808c1df3e
|
Integrate ROCm ater package for ck moe function feasibility (#2854)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Lin, Soga <soga.lin@amd.com>
|
2025-01-13 08:23:07 +00:00 |
|
|
bjmsongandroot
|
0bb0f76311
|
Support FP8 E4M3 KV Cache (#2786)
Co-authored-by: root <bjmsong@126.com>
|
2025-01-12 21:17:11 -08:00 |
|
|
Ke Bao
|
85b2e05770
|
Add int8 quant kernel (#2848)
|
2025-01-13 13:16:58 +08:00 |
|
|
Xiaoyu Zhang
|
f0e15dc6ab
|
[HotFix] fix fp8 scale load failed in tp>1 (#2837)
|
2025-01-11 14:34:26 +08:00 |
|
|
Ke Bao
|
b5fb4ef58a
|
Update modelopt config and fix running issue (#2792)
|
2025-01-08 18:04:30 +08:00 |
|
|
Lianmin ZhengandSangBin Cho rkooo567@gmail.com
|
8a6906127a
|
Improve linear.py to load sharded weights & remove the dependency of Parameters from vllm (#2784)
Co-authored-by: SangBin Cho rkooo567@gmail.com
|
2025-01-07 23:29:10 -08:00 |
|
|
Lianmin Zheng
|
bdc1acf6cd
|
Misc fix for min_p_sampling, --cuda-graph-bs (#2761)
|
2025-01-07 02:52:53 -08:00 |
|
|
Zhiyu
|
287427e2e6
|
Enable Nvidia's ModelOpt fp8 quantized models (#2535)
|
2025-01-06 14:54:52 -08:00 |
|
|
Lianmin Zheng
|
b8574f6953
|
Clean up eagle code (#2756)
|
2025-01-06 14:54:18 -08:00 |
|
|
|
c7ae474a49
|
[Feature, Hardware] Enable DeepseekV3 on AMD GPUs (#2601)
Co-authored-by: root <root@banff-cyxtera-s83-5.amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Bruce Xue <yigex@xilinx.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-01-02 16:23:19 -08:00 |
|
|
Lianmin Zheng
|
bdf946bf81
|
Support loading pre-sharded moe weights (#2716)
|
2025-01-02 15:07:37 -08:00 |
|