 Xihuai Wang
|
1afe3d0798
|
Align finish reason and stream mode in openai api (#4388)
|
2025-03-27 00:16:52 -07:00 |
|
|
laixinandAniZpZ
|
ae25d36dc6
|
[3/3] fix dsv3 awq issue (#4719)
Co-authored-by: AniZpZ <aniz1905@gmail.com>
|
2025-03-26 23:13:43 -07:00 |
|
|
Yineng Zhang
|
1099f6c974
|
bump v0.4.4.post2 (#4669)
|
2025-03-26 19:58:00 -07:00 |
|
|
Xiaoyu Zhang
|
04e3ff6975
|
Support compressed tensors fp8w8a8 (#4743)
|
2025-03-26 13:21:25 -07:00 |
|
|
Yi Pan
|
45fdf1f7f3
|
Fix shared memory OOM on sm86 GPUs. (#4797)
|
2025-03-26 10:41:53 -07:00 |
|
|
Kebe
|
d89c0e4b7e
|
Use metadata to detect version of package (#4782)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-26 00:41:43 -07:00 |
|
|
fzyzcjy
|
fa3c9e0668
|
Fix popen_launch_server wait for 20 minutes when child process exits (#4777)
|
2025-03-26 00:32:19 -07:00 |
|
|
Cheng Wan
|
0d658ac3df
|
Support recording experts workload in QWen2-MoE (#4775)
|
2025-03-26 00:29:57 -07:00 |
|
|
Thysrael
|
ced35a0649
|
fix(typo): fix reply to replay in base_attn_backend.py (#4784)
|
2025-03-26 00:19:12 -07:00 |
|
|
fzyzcjy
|
26f07294f1
|
Warn users when release_memory_occupation is called without memory saver enabled (#4566)
|
2025-03-26 00:18:14 -07:00 |
|
|
Baizhou Zhang
|
34e07a65f1
|
[Fix] Fix unexpected idx bug of Phi-3-small (#4728)
|
2025-03-25 21:33:48 -07:00 |
|
|
fzyzcjy
|
15ddd84322
|
Add retry for flaky tests in CI (#4755)
|
2025-03-25 16:53:12 -07:00 |
|
|
fzyzcjy
|
52029bd1e3
|
Fix warmup error when dp=1 (#4753)
|
2025-03-25 02:01:21 -07:00 |
|
|
fzyzcjy
|
eb934bdf3b
|
Fix test_expert_distribution failure (#4752)
|
2025-03-25 01:17:03 -07:00 |
|
|
DarkSharpness
|
ac3fae8445
|
[Feature] Support "strict" in function calling (#4310)
|
2025-03-24 22:15:25 -07:00 |
|
|
yuhsaun-tandCheng Wan
|
199bb01d00
|
Add endpoints to dump selected expert ids (#4435)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-24 21:34:19 -07:00 |
|
|
fzyzcjy
|
6b7038babd
|
Speedup warmup when DP > 1 (#4695)
|
2025-03-24 21:08:05 -07:00 |
|
|
lukecandyinfan98
|
57eec0bfbc
|
fix FlashMLA cudagraph config (#4691)
Co-authored-by: yinfan98 <1106310035@qq.com>
|
2025-03-24 21:06:58 -07:00 |
|
|
fzyzcjy
|
f01b092519
|
Super tiny fix typo (#4738)
|
2025-03-24 21:05:45 -07:00 |
|
|
Mick
|
1e86457c90
|
model: Minicpmo (#3023)
|
2025-03-24 20:08:40 -07:00 |
|
|
Ximingwang-09andximing.wxm
|
22c3702e1e
|
[Model] Support Qwen2ForSequenceClassification (#4609)
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
|
2025-03-24 19:13:44 -07:00 |
|
|
Stefan He
|
4c584fc632
|
Fix circular imports in gptq.py and unblock test explorer (#4736)
|
2025-03-24 18:07:08 -07:00 |
|
|
Ke Bao
|
77cf771ebe
|
Fix EAGLE3 for llama3.3 70b (#4716)
|
2025-03-24 17:31:19 -07:00 |
|
|
Xuchun Shang
|
8154de5a32
|
[PD] Remove invalid parameter (#4721)
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
|
2025-03-24 13:14:16 -07:00 |
|
|
Yineng Zhang
|
c11cfda07b
|
update pyproject (#4731)
|
2025-03-24 09:50:28 -07:00 |
|
|
Yuhong Guo
|
64edeb798f
|
Support dynamic version name in sglang's pyproject.toml (#4720)
|
2025-03-24 08:56:31 -07:00 |
|
|
|
5d7edc8e55
|
Support FA3 as Attention backend by using --attention-backend fa3 (#4680)
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
|
2025-03-23 23:28:11 -07:00 |
|
|
Alex Sun
|
af6535e7aa
|
[ROCm] Enable MTP (NextN) on AMD GPU (#4631)
|
2025-03-23 22:58:05 -07:00 |
|
|
c1lovez1
|
93cf7fc5cd
|
Unify variable naming: replace is_in_free_group with is_not_in_free_group (#4698)
|
2025-03-23 21:51:08 -07:00 |
|
|
Kyungmin Lee
|
2a206b22ed
|
Fix RotaryEmbedding when using Triton backend for EXAONE-3.5-2.4B (#4064)
|
2025-03-23 17:58:12 -07:00 |
|
|
Zhiqiang Xie
|
4d25305700
|
Move mem_state update into debug mode (#4525)
|
2025-03-23 00:52:27 -07:00 |
|
|
Mick
|
11577cedb7
|
refactor: bug fixes and refactor for vlm (#4661)
|
2025-03-22 22:48:49 -07:00 |
|
|
fzyzcjyandCheng Wan
|
ca75741e86
|
Support async in DeepEP (#4610)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-22 22:39:56 -07:00 |
|
|
fzyzcjy
|
c6d549e773
|
Multiple tiny code cleanups (#4608)
|
2025-03-22 22:39:11 -07:00 |
|
|
xutizhouandCheng Wan
|
c2bd094d6e
|
Optimize Permute Kernel in DeepEP (#4643)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-22 14:30:34 -07:00 |
|
|
Adarsh ShirawalmathandChayenne
|
f8f9244a61
|
[Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984)
Co-authored-by: Chayenne <zhaochen20@outlook.com>
|
2025-03-22 14:27:39 -07:00 |
|
|
Zhiqiang Xie
|
ecbfe58bb0
|
Bug fix for metrics counter (#4660)
|
2025-03-22 13:39:21 -07:00 |
|
|
Yun Dai
|
8cd4250401
|
[quantization] fix channelwise conversion with scalar weight scale (#4596)
|
2025-03-22 00:47:52 -07:00 |
|
|
mlmzandshuaills
|
f6ab4ca6bc
|
fix: fix ipython running error for Engine due to outlines nest_asyncio (#4582)
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
|
2025-03-21 19:11:15 -07:00 |
|
|
|
c7c7dbebbe
|
[PD] Release initial code (#4654)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai
|
2025-03-21 14:47:47 -07:00 |
|
|
Yuhong Guo
|
417fc72f6f
|
Align completion and chat_completion response to OpenAI API (#4637)
|
2025-03-20 22:59:04 -07:00 |
|
|
lukec
|
4c56e5dbee
|
Set deepgemm to the default value in the hopper architecture. (#4613)
|
2025-03-20 22:03:00 -07:00 |
|
|
Cheng Wan
|
7b5fc71972
|
fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640)
|
2025-03-20 21:45:07 -07:00 |
|
|
Yuhong Guo
|
b36ab493b3
|
Enable setting sglang logger from Env Variable SGLANG_LOGGING_CONFIG_PATH (#4592)
Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>
|
2025-03-20 02:10:32 -07:00 |
|
|
JieXin Liang
|
9e93ef3f8e
|
[fix] fix illegal mem access and clean up triton attention backend (#4571)
|
2025-03-20 02:01:52 -07:00 |
|
|
|
fad86a6863
|
Support n in OpenAI API completions (#3446)
Co-authored-by: Shan Yu <shanyu1@g.ucla.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: chuyue sun <chuyue@lmsys.us-northcentral1-a.compute.internal>
|
2025-03-20 13:46:46 +08:00 |
|
|
strgrbandZhang Kaihong
|
df7014a8d2
|
avoid cudaStreamSynchronize in DeepSeekV2AttentionMLA (#4577)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-19 10:02:26 -07:00 |
|
|
JieXin Liang
|
4942074174
|
[fix] fix initialization of _ENABLE_TORCH_INFERENCE_MODE (#4549)
|
2025-03-19 09:57:59 -07:00 |
|
|
|
b6944f97a6
|
Support FlashMLA backend cuda graph (#4514)
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
Co-authored-by: ispobock <ispobaoke@163.com>
|
2025-03-19 08:25:34 -07:00 |
|
|
|
f44db16c8e
|
[Feature] Integrate DeepEP into SGLang (#4232)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
|
2025-03-19 08:16:31 -07:00 |
|