Lianmin Zheng
|
b3f3d610fd
|
Do not use FA3 for mistral (#6379)
|
2025-05-17 19:47:34 -07:00 |
|
ishandhanani
|
b6909aa223
|
fix: allow launch_dummy_health_check_server to start inside of running asyncio loop (#6330)
|
2025-05-17 18:32:41 -07:00 |
|
fzyzcjy
|
fd08c04821
|
Support custom DeepEP tuning config (#6257)
|
2025-05-17 17:09:42 -07:00 |
|
Lifu Huang
|
3cf1473a09
|
Use monotonic clock for interval measurement (#6211)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-17 16:49:18 -07:00 |
|
fzyzcjy
|
f194e14fb7
|
Reduce MoE memory usage (#6147)
|
2025-05-15 09:38:28 -07:00 |
|
JieXin Liang
|
9a405274e2
|
[misc] remove redundant platform codes (#6298)
|
2025-05-15 00:51:30 -07:00 |
|
Lianmin Zheng
|
e8e18dcdcc
|
Revert "fix some typos" (#6244)
|
2025-05-12 12:53:26 -07:00 |
|
Ying Sheng
|
bad7c26fdc
|
[PP] Fix init_memory_pool desync & add PP for mixtral (#6223)
|
2025-05-12 12:38:09 -07:00 |
|
applesaucethebun
|
d738ab52f8
|
fix some typos (#6209)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-05-13 01:42:38 +08:00 |
|
JieXin Liang
|
e9bebafb19
|
[fix] remove mixtral from is_fa3_default_architecture (#6191)
|
2025-05-11 00:15:54 -07:00 |
|
Baizhou Zhang
|
73600673bb
|
Clean logs for DeepSeek-V3 launching (#6079)
|
2025-05-07 18:54:50 -07:00 |
|
Song Zhang
|
00c2c1f08b
|
[Feature] Support for Ascend NPU backend (#3853)
Signed-off-by: Song Zhang <gepin.zs@antgroup.com>
Co-authored-by: 22dimensions <waitingwind@foxmail.com>
|
2025-05-06 20:32:53 -07:00 |
|
Junrong Lin
|
357fb2dba5
|
fix: fix broadcast_pyobj breaking VerlEngine (#5997)
|
2025-05-04 13:15:53 -07:00 |
|
Ying Sheng
|
11383cec3c
|
[PP] Add pipeline parallelism (#5724)
|
2025-04-30 18:18:07 -07:00 |
|
Yineng Zhang
|
9a6ad8916d
|
chore: upgrade sgl-kernel 0.1.1 (#5933)
|
2025-04-30 16:13:30 -07:00 |
|
Yineng Zhang
|
663037a7a0
|
feat: update is_fa3_default_architecture (#5854)
|
2025-04-28 11:53:22 -07:00 |
|
Lianmin Zheng
|
daed453e84
|
[CI] Improve github summary & enable fa3 for more models (#5796)
|
2025-04-27 15:29:46 -07:00 |
|
Lianmin Zheng
|
621e96bf9b
|
[CI] Fix ci tests (#5769)
|
2025-04-27 07:18:10 -07:00 |
|
Yineng Zhang
|
04f2abcb34
|
fix: gemma 3 not use softcap (#5622)
|
2025-04-22 01:16:08 -07:00 |
|
Lianmin Zheng
|
1343200299
|
Clean up mem settings (#5610)
|
2025-04-21 17:19:00 -07:00 |
|
JieXin Liang
|
c2942907d5
|
[feature] enable pre compile jit deep_gemm (#5580)
|
2025-04-21 16:52:53 -07:00 |
|
Liangsheng Yin
|
e69a219074
|
Enhance GPU memory settings (#5604)
|
2025-04-21 15:15:00 -07:00 |
|
Lucius
|
3b6d539f63
|
[Fix] Enhance DP Attention for IPv6 Compatibility (#4937)
|
2025-04-21 00:44:11 -07:00 |
|
lambert0312
|
c44f2869c9
|
Modify metrics service endpoint (#3443)
|
2025-04-21 00:35:38 -07:00 |
|
fzyzcjy
|
685d8980c3
|
Tiny add warning when cannot recognize bool env var (#5348)
|
2025-04-20 23:11:29 -07:00 |
|
Qingquan Song
|
188f0955fa
|
Add Speculative Decoding Eagle3 topk > 1 (#5318)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
|
2025-04-20 22:58:28 -07:00 |
|
JieXin Liang
|
97cb762bb6
|
[misc] remove is_cuda_available (#5319)
|
2025-04-20 18:16:51 -07:00 |
|
Yineng Zhang
|
66c0ff9e31
|
fix: use fa3 for gemma2 (#5586)
|
2025-04-20 17:02:09 -07:00 |
|
fzyzcjy
|
613b197e57
|
Remove one kernel in per_tensor_quant_mla_fp8 (#5549)
|
2025-04-19 15:08:15 -07:00 |
|
Lianmin Zheng
|
177320a582
|
Clean up imports (#5467)
|
2025-04-16 15:26:49 -07:00 |
|
Yineng Zhang
|
fa909dc3c4
|
feat: update model_specific_adjustment (#5344)
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
|
2025-04-15 14:45:15 -07:00 |
|
shangmingc
|
ffde65a094
|
[PD] Fix dynamic port support and MLA buffer for Mooncake (#5415)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
|
2025-04-15 19:29:31 +08:00 |
|
lambert0312
|
471650dee0
|
Fix broadcast use cuda device lead to memory capacity unbalanced (#5416)
|
2025-04-15 02:47:26 -07:00 |
|
Yineng Zhang
|
39e411385c
|
fix #5322 (#5359)
|
2025-04-13 13:57:36 -07:00 |
|
mlmz
|
8311b07fb9
|
Fix: Ensure tensors for dist.broadcast match NCCL backend device (#5322)
|
2025-04-12 22:50:37 -07:00 |
|
Yineng Zhang
|
7d3b7c87f5
|
fix: determine if flashinfer is installed (#5336)
|
2025-04-12 19:59:13 -07:00 |
|
tianlian yi
|
bc92107b03
|
Support server based rollout in Verlengine (#4848)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Jinn <47354855+jhinpan@users.noreply.github.com>
|
2025-04-12 10:07:52 -07:00 |
|
Yineng Zhang
|
57de7c6b5f
|
feat: use fa3 mla by default on hopper (#5210)
Co-authored-by: yundai424 <yundai424@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
|
2025-04-12 01:09:25 -07:00 |
|
fzyzcjy
|
86a876d883
|
Optimize topk operation in llama4 (#5128)
|
2025-04-09 02:50:22 -07:00 |
|
JieXin Liang
|
a995a773a0
|
[fix] remove cuda_device_count_stateless (#5060)
|
2025-04-04 00:18:26 -07:00 |
|
fzyzcjy
|
8e10fec9a8
|
Small refactor DeepEPMode to clean up code a bit (#4992)
|
2025-04-03 02:56:44 -07:00 |
|
XinyuanTong
|
9eb49e878b
|
[VLM RLHF] Take Image input for verl vlm rollout (#4915)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: GeLee <leege233@gmail.com>
|
2025-04-01 20:03:17 -07:00 |
|
Mick
|
5cb552b1d4
|
refactor: multimodal data (#4754)
|
2025-03-31 09:57:51 -07:00 |
|
Lianmin Zheng
|
9adf178cc2
|
Fix 2-gpu CI test and suppress some warnings (#4930)
|
2025-03-30 12:51:44 -07:00 |
|
fzyzcjy
|
8690c40bb0
|
Improve stack trace of retry errors (#4845)
|
2025-03-29 08:21:31 -07:00 |
|
rongfu.leng
|
f9970bd1af
|
fix: when use SGLANG_PORT this env,port is str (#4528)
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
|
2025-03-27 21:46:06 -07:00 |
|
Vincent
|
e2e2ab70e0
|
IPv6 support (#3949)
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-03-27 21:42:13 -07:00 |
|
Mick
|
1e86457c90
|
model: Minicpmo (#3023)
|
2025-03-24 20:08:40 -07:00 |
|
lukec
|
4c56e5dbee
|
Set deepgemm to the default value in the hopper architecture. (#4613)
|
2025-03-20 22:03:00 -07:00 |
|
Yuhong Guo
|
b36ab493b3
|
Enable setting sglang logger from Env Variable SGLANG_LOGGING_CONFIG_PATH (#4592)
Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>
|
2025-03-20 02:10:32 -07:00 |
|