chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 15 Packages Projects Releases Wiki Activity
11,116 Commits 1 Branch 0 Tags
47cb9b9d116d1c5c112e894110ec94de80f2db6b
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Liangsheng Yin d9f5c2179c ci(slash-cmd): allow write-permission users to /rerun-ut on fork PRs (#21121) 2026-03-22 00:45:48 -07:00
Liangsheng Yin 1e97864d75 ci(slash-cmd): allow repo write-permission users to /rerun-ut (#21120) 2026-03-22 00:32:29 -07:00
Liangsheng Yin 47d36d73f7 Update write-sglang-test skill: CUDA-only for common tests + prefer mock (#21119) 2026-03-21 22:54:43 -07:00
Liangsheng Yin ff921a2ed3 Update cron schedule for PR test workflow (#21090) 2026-03-21 01:03:45 -07:00
Liangsheng Yin 9614271ae4 [Test] Fix flaky multi-instance memory occupation test (#21074) 2026-03-20 22:19:03 -07:00
Liangsheng Yin 6ad78b008b Consolidate streaming session tests (4 files → 3, fewer server starts) (#20961) 2026-03-19 19:13:28 -07:00
Liangsheng Yin 8d22efdeea [Tiny Fix] Fix IS_BLACKWELL env var empty string warning in rerun-ut workflow (#20957) 2026-03-19 17:24:35 -07:00
Liangsheng Yin 2b53e660de Simplify streaming session logprob handling (#20955) 2026-03-19 17:09:40 -07:00
Liangsheng Yin 8b9482e665 fix(dp-attn): consistent overlap disable decision across DP ranks (#20853) 2026-03-18 15:16:39 -07:00
Liangsheng Yin 4d3976b6c5 [HiCache] Check in-flight async ops in is_fully_idle() before attach/detach (#20746) 2026-03-17 17:28:26 -07:00
Liangsheng Yin 5270a06488 [Disagg] Fix health check false-positive in disagg is_fully_idle (#20756) 2026-03-17 17:18:54 +08:00
Liangsheng Yin f0458e0b49 [Utils] Move network/socket utilities from common.py to network.py (#20646) 2026-03-15 20:35:24 -07:00
Liangsheng Yin d852f26cb6 Fix dual-stack socket handling: IPV6_V6ONLY, IPv4-first, is_port_available all-family check (#20643) 2026-03-15 17:17:23 -07:00
Liangsheng YinandClaude Opus 4.6 c2245d9b55 ci: add runner dropdown to rerun-ut workflow dispatch (#20645) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-15 16:40:14 -07:00
Liangsheng Yin 116aef8504 [Test] Move embedding tests into test/registered/embedding/ and unit/ (#20642) 2026-03-15 14:48:43 -07:00
Liangsheng Yin 8397a8a301 [CI] Extract wait-for-jobs composite action and gate on stage-a-cpu-only (#20641) 2026-03-15 13:52:48 -07:00
Liangsheng Yin fc7f9c1de7 Rename --stream-output to --incremental-streaming-output (#20614) 2026-03-14 23:22:33 -07:00
Liangsheng Yin f605612b87 [HTTP] Fix /GET HTTP route when ollama endpoint is not set. (#20494) 2026-03-12 20:54:32 -07:00
Liangsheng YinandClaude Opus 4.6 5f1bfb0d28 [Security] Fix /rerun-ut bypassing run-ci gate for fork PRs (#20424) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-12 02:24:29 -07:00
Liangsheng Yin 61cad15d28 [Utils] Add NetworkAddress abstraction for IPv6-safe address handling (#20306) 2026-03-11 00:07:37 -07:00
Liangsheng Yin 50953aea8d [Scheduler] Unify idle checks into is_fully_idle() and fix weight update test (#20296) 2026-03-10 17:50:23 -07:00
Liangsheng Yin ac07a6d439 Revert "[Scheduler] Decouple maybe_send_health_check_signal from process_batch_result" (#20259) 2026-03-10 01:58:48 -07:00
Liangsheng YinandClaude Opus 4.6 208f1428e9 [Scheduler] Decouple maybe_send_health_check_signal from process_batch_result (#20227) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-09 21:18:56 -07:00
Liangsheng Yin ffb4b6f4c1 [Core] Replace server_args mutation hack with explicit MemoryPoolConfig for draft worker init (#20183) 2026-03-09 11:45:54 -07:00
Liangsheng Yin 2ef00383ab [Core] Refactor init_memory_pool into composable resolution helpers (#20142) 2026-03-08 21:46:27 -07:00
Liangsheng YinandClaude Opus 4.6 7105bf3782 [Bug] Fix missing TTFT histogram for single-batch requests (#20122) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-08 15:18:51 -07:00
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 29f3a5396e [Minor] Add SessionSlot.is_holding_kv property for readability (#20120) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-08 03:25:13 -07:00
Liangsheng YinYilong ZhaoAurick Qiao
 		36b557d2c9 Fix streaming session with paged KV cache (SWA/MLA) (#20070) 
Co-authored-by: Yilong Zhao <74357408+happierpig@users.noreply.github.com>
Co-authored-by: Aurick Qiao <6137920+aurickq@users.noreply.github.com>
 2026-03-08 03:00:32 -07:00
Liangsheng Yin d02c515ee8 Decouple scheduler log printing from metrics collection (#20107) 2026-03-07 22:09:10 -08:00
Liangsheng Yin 69ec15ad74 [CI] Re-enable streaming session tests with correctness and stress coverage (#20084) 2026-03-07 18:18:11 -08:00
Liangsheng YinandClaude Opus 4.6 ddcecdea49 [Core] Unify max_num_reqs dp_size division for pool sizing (#20063) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-06 16:12:59 -08:00
Liangsheng Yin 604db4471d [Core] Clarify memory variable naming in model runner (#20060) 2026-03-06 14:00:46 -08:00
Liangsheng Yin 7a6cf0e9ba [Core] Extract _calculate_mamba_ratio and _init_pools from init_memory_pool (#20058) 2026-03-06 13:37:22 -08:00
Liangsheng Yin 861d78635f [CI] remove itl testing due to unstable networking (#19904) 2026-03-04 16:32:17 -08:00
Liangsheng Yin 33c92732f4 [Triton] Use dynamic loop bound in alloc_extend_kernel (#19898) 2026-03-04 16:15:58 -08:00
Liangsheng Yin c7ffbf25e9 [CI] Fix rerun-ut workflow: add DeepEP install, RDMA env, Blackwell detection (#19803) 2026-03-03 15:17:16 -08:00
Liangsheng Yin 1135e214b3 [CI] support /rerun-ut command in slash handler (#19800) 2026-03-03 14:10:49 -08:00
Liangsheng Yin 7a2d3df96f Apply default stream to priority 0 in scheduling. (#16438) 2026-03-03 00:05:27 -08:00
Liangsheng Yin 5acb45cf32 [Session] Extract SessionController and clean up session logic in Scheduler (#19547) 2026-02-28 19:47:44 -08:00
Liangsheng Yin ac400cb7bb [CLI] Add --model-type override and keep launch_server supported (#19523) 2026-02-27 18:16:31 -08:00
Liangsheng Yin e08ef06758 [Session] Gate streaming sessions with --enable-streaming-session and spec v2 guard (#19531) 2026-02-27 18:14:55 -08:00
Liangsheng Yin 1a32c0db4d [Minor] Rename misleading chunked to reusing in ReqToTokenPool.alloc() (#19465) 2026-02-26 16:24:03 -08:00
Liangsheng Yin c60dcc40bb [Logging] Guard log_prefill_stats against idle batches in disagg prefill (#19361) 2026-02-25 13:31:52 -08:00
Liangsheng YinandYangmin Li 76d5410e01 [Disagg] Fix decode querying unregistered dp_rank when prefill dp_size is 1 (#19305) 
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
 2026-02-24 22:52:39 -08:00
Liangsheng YinandCursor ab0f608788 [PD-Disagg] Fix bootstrap server race condition when prefill workers not yet registered (#19288) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 20:22:16 -08:00
Liangsheng YinandRatish P 539f772f54 [PD-Disagg] Fully support external DP dispatch w/ PD-disaggregation mode. (#19268) 
Co-authored-by: Ratish P <114130421+ratish1@users.noreply.github.com>
 2026-02-24 19:58:01 -08:00
Liangsheng Yin 5e80027ac7 [PD-Disagg] Support FAKE transfer backend for more cases (#19215) 2026-02-23 20:50:43 -08:00
Liangsheng Yin feb041f4e5 [PD-Disagg] Improve type hints across all conn.py (#19208) 2026-02-23 19:44:23 -08:00
Liangsheng YinandShangming Cai ea7ef63e6d [PD-Disagg] Deduplicate common KVManager methods into CommonKVManager (#19205) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-02-23 18:34:21 -08:00
Liangsheng Yin d160c5b9bb [PD-Disagg] Unify prefill info data transition flow, all with PrefillServerInfo (#19195) 2026-02-23 16:15:20 -08:00
Liangsheng Yin 9fac90d85d [CI] Tiny enhance the dp attention load blance benchmark (#19194) 2026-02-23 14:33:32 -08:00
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2aa3fe394e [CI] fix the teardown output of disaggregation test (#19193) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-23 12:41:03 -08:00
Liangsheng YinandChang Huaixin 2274bfebb1 [PD-Disagg] Support query dp rank from bootstrap server. (#19168) 
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
Co-authored-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
 2026-02-23 10:59:30 -08:00
Liangsheng Yin 4653939cda Revert "[jit kernel] Support per_token_group_quant_8bit jit kernel" (#19131) 2026-02-22 07:54:24 +08:00
Liangsheng Yin 1f2da824dd [Benchmark] Remove re-exports from bench_serving.py (#19130) 2026-02-21 14:30:30 -08:00
Liangsheng Yin 3fb457b103 Tiny rename cuda_graph tests to piecewise_cuda_graph (#19128) 2026-02-21 13:46:01 -08:00
Liangsheng Yin db34c1cbfb Tiny remove duplicate coredump env injection (#19023) 2026-02-19 13:26:30 -08:00
Liangsheng Yin 5ff5aa6923 [spec v2]Fix torch gc of future indices (#18958) 2026-02-19 11:38:25 -08:00
Liangsheng Yin 2d85f01d43 Revert "Fix generated-shared-prefix bench_serving" (#18956) 2026-02-17 20:43:55 -08:00
Liangsheng Yin 83a475e8d7 feat: add cuda core dump CI warpper (#18909) 2026-02-17 14:49:26 -08:00
Liangsheng Yin dcea74d63f Add timeout abort kits for normal / eagle. (#18815) 2026-02-13 17:57:30 -08:00
Liangsheng Yin 4474fb98b4 [PD-Disagg] Fix double free when prebuilt batch is aborted. (#18822) 2026-02-13 17:46:35 -08:00
Liangsheng Yin e6f7a372ef Rename request timeout env vars for waiting/running stages (#18766) 2026-02-12 22:58:40 -08:00
Liangsheng Yin d29e331491 [Spec] Move forward timeout before verify to fix Eagle v1 filter mismatch (#18760) 2026-02-12 20:58:34 -08:00
Liangsheng Yin 3404dda592 Try fix the max-parallel for maunally triggered test again. (#18686) 2026-02-11 20:08:25 -08:00
Liangsheng Yin b2d7fd5c87 fix the max-parallel for /rerun-stage (#18658) 2026-02-11 19:06:51 -08:00
Liangsheng Yin 10c6bee74f List more CI runs for pr-test (#18650) 2026-02-11 18:36:45 -08:00
Liangsheng Yin cd90346a2b Add cache hit rate UT (#18566) 2026-02-10 21:27:41 -08:00
Liangsheng Yin 50f74285e9 Tiny fix regex warning (#18592) 2026-02-10 21:07:53 -08:00
Liangsheng Yin 93fca0bbc3 Fix wrong prefill log. (#18570) 2026-02-10 15:54:03 -08:00
Liangsheng Yin 2825f5d8e0 Tiny fix wrong metric collect key name forward_prefill -> forward_extend (#18506) 2026-02-09 16:23:41 -08:00
Liangsheng Yin 875ad6cf35 Tiny rename for spec related fileds. (#18468) 2026-02-09 00:10:39 -08:00
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 8278ef0e68 Pass GPU ids to kill specified devices in script. (#17840) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-27 13:52:55 -08:00
Liangsheng Yin 85d077f44d Introduce global alloc_len_per_decode & clean check decode memory (#15115) 2026-01-26 10:26:20 -08:00
Liangsheng Yin e2c8a50b38 fix grammar timeout sync across tp ranks. (#16898) 2026-01-14 10:26:31 +08:00
Liangsheng YinandDarkSharpness 2b3791ed37 Fix wrong kernel selection for int32/int64 indices (#16912) 
Co-authored-by: DarkSharpness <2040703891@qq.com>
 2026-01-12 17:26:57 +08:00
Liangsheng Yin 5b7bed7ca4 Decouple grammar logic out of scheduler. (#16820) 2026-01-12 10:52:42 +08:00
Liangsheng Yin 934ae89abe Tiny fix typo minxin -> mixin (#16908) 2026-01-12 00:36:45 +08:00
Liangsheng Yin cf1426a7b7 [CI] reapply max-parallel in stage-b-test-small-1-gpu (#16906) 2026-01-11 23:17:38 +08:00
Liangsheng Yin 09e2571e2e Clarify the meaning of cpu_group / entry_rank when dp + tp is enabled. (#16876) 2026-01-11 13:04:43 +08:00
Liangsheng Yin 78c41758ad [hot fix ci] Hot fix for the unregistered. (#16877) 2026-01-11 01:24:32 +08:00
Liangsheng Yin 3c358736d1 Enhance test for dp-attention + constrained decoding. (#16849) 2026-01-10 16:49:10 +08:00
Liangsheng Yin e6d40bff81 Revert "feat: reduce constrained-decoding overhead in TP" (#16845) 2026-01-10 11:39:38 +08:00
Liangsheng Yin 7066711529 [ci hot fix] fix global server args init for mamba ut (#16821) 2026-01-10 02:03:32 +08:00
Liangsheng Yin 87ee6b5eaa Tiny remove auto-triggering for list-active-pr-runs. (#16778) 2026-01-09 13:16:23 +08:00
Liangsheng Yin f7c1d24b18 List ci workflows status. (#16774) 2026-01-09 13:02:12 +08:00
Liangsheng Yin 75da784d48 Tiny simplify draft worker init. (#16446) 2026-01-09 11:20:29 +08:00
Liangsheng Yin 77d3566555 Tiny fix wording about CI preemption. (#16773) 2026-01-09 11:08:17 +08:00
Liangsheng Yin 064ae3419e Adjust the cancel PR workflows for better preemption. (#16749) 2026-01-09 03:25:41 +08:00
Liangsheng Yin 8867d24879 Tiny adjust cancel PR workflow. (#16697) 2026-01-08 11:13:46 +08:00
Liangsheng Yin a7fd810842 Allow editable install without .git with add fallback version in pyproject.toml (#16435) 2026-01-05 11:17:20 +08:00
Liangsheng Yin 229938805f Make personal configs optional in SGLang's official docker image. (#16365) 2026-01-04 12:27:13 +08:00
Liangsheng Yin db6b51a838 Add CI_PERMISSION sort hook. (#16200) 2025-12-31 12:15:15 +08:00
Liangsheng Yin ba67e006a7 Refactor speculative algorithm registry. (#16168) 2025-12-31 01:24:22 +08:00
Liangsheng Yin 4de59d83a1 Reduce stages of pr-test from 4 to 3. (#16173) 2025-12-30 22:21:48 +08:00
Liangsheng Yin b6871ba7c9 [CI] set max-parallel to 4 by default. (#16154) 2025-12-30 17:27:48 +08:00
Liangsheng Yin c2601f0d21 Fix wrong assigning extend_input_len_per_req with eagle. (#16129) 2025-12-30 15:51:53 +08:00
Liangsheng Yin 3946dad6fa Add attack204 into CI_PERMISSION (#16131) 2025-12-30 13:55:00 +08:00
Liangsheng Yin ac03ec0851 Add PR review process into template. (#16133) 2025-12-30 13:45:56 +08:00
Liangsheng Yin a435f55d18 Tiny print launch command with shlex (#16010) 2025-12-29 11:26:46 +08:00