chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
7,655 Commits 1 Branch 0 Tags
02f1e81e2defa70b5bcdc59eb878dec415fdfd12
Commit Graph

7655 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yuhao Yang
02f1e81e2d Revert "fix: checking if tokenizer is in cache before downloading from HF" (#14808) 2025-12-10 01:14:35 -08:00
Prozac614
908c7186af [diffusion] CI: Add LoRA support to diffusion server configuration and test cases (#14697) 2025-12-10 16:51:47 +08:00
Yuan Luo
03836d85d2 [GLM-4.6V] Support Pipeline Parallelism for GLM-4.6V & GLM-4.1V (#14720) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-10 16:40:12 +08:00
Mick
87dbdddc93 [diffusion] profile: early exit when enough steps are captured to reduce the size of the trace file (#14803) 2025-12-10 16:11:22 +08:00
b8zhong
56e5c07424 fix b200 fa4 ci (#14788) 2025-12-10 00:03:43 -08:00
yrk111222
6c9c8da64d fix: add missing logic for SGLANG_USE_MODELSCOPE variable (#14794) 2025-12-10 15:49:34 +08:00
Peng Zhang
21028b5507 [RL] support weight reload for low-bit rollout (#9650) 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-12-10 15:44:01 +08:00
b8zhong
b0a25d0913 fix b200 ci (#14786) 2025-12-09 23:08:41 -08:00
Yuhao Yang
793c98afaf handling incomplete rope_scaling config ci after transformers upgrade (#14784) 2025-12-09 22:56:16 -08:00
TomerBN-Nvidia
b1cbfce612 fix server args bug (#14725) 2025-12-09 21:05:05 -08:00
Alison Shao
b0f531ad28 Fix VLM accuracy thresholds for nightly tests (#14777) 2025-12-09 20:59:23 -08:00
Alison Shao
01835998e1 fix: race condition between validation and download locks (#14761) 2025-12-09 20:36:54 -08:00
Lianmin Zheng
4285e99da7 [Auto Sync] Update data_parallel_controller.py, detokenizer... (20251209) (#14759) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-09 18:38:38 -08:00
ShawnY112358
f077436831 [fix] Fix issues for in-flight weight updates (#14064) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-09 18:12:19 -08:00
Alison Shao
5e8f544d1b Disable 8-gpu-b200 runner in PR tests (#14768) 2025-12-09 17:47:54 -08:00
Douglas Yang
c8d74feb06 fix: adding rate limit warning at verify token permission stage (#14756) 2025-12-09 17:21:04 -08:00
Liangsheng Yin
cbc7dcdaa7 Re-add the API serving timing metrics. (#14744) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
Co-authored-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-10 10:17:48 +09:00
MingxuZh
a6dc7d2932 [ci]: Enable the new hf API (#14687) 2025-12-09 17:07:51 -08:00
Simo Lin
390406c46c [model-gateway] release gateway 0.2.4 (#14763) 2025-12-09 16:34:29 -08:00
Ethan (Yusheng) Su
0c63fb9420 [Feature] Add LoRA support for embedding layers (#14177) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Beichen-Ma <bm685@cornell.edu>
 2025-12-09 15:53:33 -08:00
Simo Lin
9ad02b799d [Perf] Optimize radix tree for cache-aware load balancin (#14758) 2025-12-09 15:35:29 -08:00
Lianmin Zheng
18bd8e8d6d Improve CI by trying a warmup before unit tests (#14669) 2025-12-09 15:17:59 -08:00
Lianmin Zheng
036e64dafa move multi-item scoring functions in tokenizer manager into a separate file (#14740) 2025-12-09 14:47:06 -08:00
Douglas Yang
7c6fb3aa2d fix: making rate limit a warning instead of error (#14753) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-09 13:41:09 -08:00
Douglas Yang
8b0b6a45c8 fix: checking if tokenizer is in cache before downloading from HF (#14698) 2025-12-09 13:08:07 -08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Simo Lin
73df7a4e8d [SMG] perf: optimize tokenizer for reduced CPU and memory overhead (#14752) 2025-12-09 11:41:13 -08:00
Simo Lin
8b98bb768c [model-gateway] optimize core modules (#14751) 2025-12-09 10:54:07 -08:00
Yuhao Yang
15bc8cbd74 fix rope parameter initialization error caused by transformers v5.0 update (#14745) 2025-12-09 10:51:26 -08:00
yudian0504
9496f12d00 [Model] Add PaddleOCR-VL Model Support (#12953) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-09 10:16:02 -08:00
Lianmin Zheng
ab0048793c Revert "[Feat] Add received_time in serving_base" (#14743) 2025-12-09 06:52:27 -08:00
blahblah
6ec77680df [diffusion] feat: support comparing batch perf (#14738) 
Co-authored-by: shuxiguo <shuxiguo@meituan.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-09 22:48:52 +08:00
Alison Shao
13680e5542 [Test] Skip STANDALONE speculative decoding tests for different hidden sizes (#14733) 2025-12-09 22:21:35 +09:00
yrk111222
98c430e114 fix: prevent HugginqFace access when SGLANG_USE_MODELSCOPE is enabled (#12039) 2025-12-09 21:02:05 +08:00
zhanghaotong
fe7f91ef82 [Feat] Add received_time in serving_base (#13432) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-09 22:00:24 +09:00
kun-llfl
cef5ba65b1 [Bugfix] Fix environ error in scheduler_runtime_checker_mixin.py (#14461) 
Signed-off-by: Kun(llfl) <i@imux.top>
 2025-12-09 20:33:41 +08:00
Xiaoyu Zhang
53d170883a Add fuse_marlin_moe test to ci and add new ep test (#14686) 2025-12-09 20:17:38 +08:00
Rain Jiang
f0e948a0f1 fix the deepep 8 gpu unit test (#14601) 2025-12-09 01:40:09 -08:00
Alison Shao
9a426fc5ef [CI] Move mistral large 3 basic to nightly (#14622) 2025-12-09 00:28:44 -08:00
Yineng Zhang
66772aa2b4 chore: add code owners for deepseek_v2.py (#14714) 2025-12-08 23:24:06 -08:00
Yineng Zhang
b626334475 fix: make override DeepseekV2Model work (#14707) 2025-12-08 23:11:18 -08:00
b8zhong
0f8bd55f3e [CI] Fix Llama 3.1 8B FP4 CI (#14699) 2025-12-08 22:27:15 -08:00
fzyzcjy
da3dc497b0 Fix dp-aware incompatible with completions and chat completions APIs (#14647) 2025-12-09 13:24:53 +08:00
fzyzcjy
817daba062 Tiny extract select_worker_min_load (#14648) 2025-12-08 21:22:10 -08:00
Simo Lin
af60cad05d [ci][smg] fix docker release ci and add it to pr test (#14683) 2025-12-08 20:56:38 -08:00
Lianmin Zheng
ce4e836be5 Add per-request decode tp size (#14678) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-12-08 20:24:31 -08:00
Yineng Zhang
0e0b0c0566 Revert "[Bug] fix not desired disable fused share experts caused by r… (#14676) 2025-12-08 20:06:52 -08:00
Alison Shao
e6f0ddda44 [CI] Migrate Eagle 1-GPU tests to test/registered/ (#14529) 2025-12-09 12:56:36 +09:00
fzyzcjy
af20657cd4 Tiny support sgl-router http response status code metrics (#14689) 2025-12-08 19:54:35 -08:00
Alison Shao
08da4c2618 [Bugfix] Fix KeyError for Mistral-Large-3 rope_scaling config (#14627) 2025-12-08 19:16:00 -08:00