laoyao0822
2c94b8de23
Ground HiCache backup overlap policy in measured tails
...
The per-layer backup path did not explain end-to-end performance by inspection alone, so add a focused benchmark that compares all-layer tail backup against per-layer forward-overlap backup under the same page_first_direct layout. The benchmark reports total latency, over-forward-only latency, submit overhead, and winner so short-extend fallback decisions are based on measured cost rather than request-size intuition.
Constraint: CUDA benchmark execution is remote-only; this local commit only records the benchmark driver.
Rejected: Adding more online logs for this question | the user asked for benchmark evidence and logs would perturb scheduler behavior.
Confidence: high
Scope-risk: narrow
Directive: Do not infer production policy from per_extra alone; per-layer CPU submission can be hidden inside the forward timing interval, so total latency is the safer comparison metric.
Tested: Local py_compile benchmark/hicache/bench_cp_hicache_backup_overlap.py.
Tested: Remote benchmark logs generated under /mnt/beegfs/cjy/cp_hicache_backup_overlap_bench_20260527_*.log.
Not-tested: Full SGLang test suite; benchmark is standalone and CUDA execution was remote-only.
2026-05-28 03:43:05 +08:00
leavelet
2ae0237d3e
fix(deepep): tuning and enhance deepep on single node
...
feat(deepep): enhance single node deepep
2026-04-08 05:36:31 +08:00
Lianmin Zheng
104b10f70a
refactor: consolidate is_in_ci (jit_kernel, sgl-kernel benchmarks, tests) ( #21009 )
2026-03-20 05:55:36 -07:00
cs-cat
22e378af86
Fix result writer in tuning_block_wise_kernel.py, and add FP8 kernel config for L40 ( #20368 )
...
Signed-off-by: cs-cat <118669451+cs-cat@users.noreply.github.com >
2026-03-20 09:28:54 +08:00
6b8a6545b2
Add Mistral Small 4 (Pixtral) support ( #20708 )
...
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
Co-authored-by: Alex Nails <alexnails@radixark.ai >
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com >
Co-authored-by: dbari <dbari@users.noreply.github.com >
2026-03-18 14:15:32 -07:00
Yuan Luo
9c87e137ee
[GDN] Support GDN packed decode ( #20627 )
2026-03-18 13:20:07 +08:00
Xiaoyu Zhang
25e38216b6
[kernel slimming] Clean many useless sgl-kernel deprecated kernels ( #20277 )
2026-03-14 16:45:54 +08:00
Chongchong Tian and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
70d4aabe42
Add CLI args to conveniently support tuning more models ( #12922 )
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-12 23:10:55 -07:00
e29305c120
[GDN] Add benchmark for sglang gdn prefill ( #20428 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
Co-authored-by: Kaixi Hou <kaixih@nvidia.com >
2026-03-12 22:25:02 +08:00
Mook
abc672e717
[Benchmark] use flashinfer bench_gpu_time instead of triton do_bench ( #20305 )
2026-03-12 04:04:30 +00:00
Yuan Luo and luoyuan.luo
751c454099
Add DeepSeek3.2 and GlmMoeDsa into moe tune ( #18876 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-03-10 17:12:58 +08:00
Mohammad Miadh Angkad
1b76eb9361
[Doc] Update version references and add automation ( #18409 )
2026-03-04 09:51:46 -08:00
Kangyan-Zhou and Claude Opus 4.6
dc92f88a21
Enhance bench_multiturn.py with OpenAI API support and richer metrics ( #19724 )
...
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-03 13:48:04 -08:00
RoyWang
a1ef8e2cc0
[AMD] optimize Kimi K2.5 fused_moe_triton performance by tuning ( #19228 )
2026-02-26 11:50:13 -08:00
a0a8f1473c
[Benchmark] Fix generated_shared_prefix attribute naming and remove args dependency ( #19363 )
...
Co-authored-by: Alison Shao <alisonshao@Mac.attlocal.net >
Co-authored-by: sglang-bot <sglangbot@gmail.com >
2026-02-25 18:45:54 -08:00
a55f658835
[Misc] Normalize --host parameter to use plain hostname without scheme ( #19309 )
...
Co-authored-by: 墨楼 <huangzhilin.hzl@antgroup.com >
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-02-25 00:37:24 -08:00
Hubert Lu and yctseng0211
17b0affbdf
[AMD] Support --enable-aiter-allreduce-fusion on AMD GPUs ( #13747 )
...
Co-authored-by: yctseng0211 <yctseng@amd.com >
2026-02-24 23:11:55 -08:00
Ratish P and Liangsheng Yin
ae6f6e1495
[Refactor] Benchmark: Add typed DatasetArgs/Loader registry and CPU dataset unit tests ( #19147 )
...
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
2026-02-24 12:22:01 -08:00
Alec Leng
38f25e802d
Fix/deepseek readme link ( #19258 )
2026-02-24 10:46:56 -08:00
581bf53e03
Whisper model support & /v1/audio/transcriptions endpoint & benchmark ( #16983 )
...
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
Co-authored-by: MahmoudAshraf97 <hassouna97.ma@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-23 17:28:37 -08:00
Liangsheng Yin
1f2da824dd
[Benchmark] Remove re-exports from bench_serving.py ( #19130 )
2026-02-21 14:30:30 -08:00
satyamk7054 and Satyam Kumar
355127c2e9
Fix benchmark_sglang_fused_moe_triton.py ( #18940 )
...
Co-authored-by: Satyam Kumar <satyamk@linkedin.com >
2026-02-17 17:25:37 -05:00
SoluMilken
07a24f1a38
update pre-commit config ( #18860 )
2026-02-16 00:18:31 +08:00
Ke Bao
a9d59776cc
Enhence gsm8k test ( #18791 )
2026-02-13 18:08:57 +08:00
Liangsheng Yin
cd90346a2b
Add cache hit rate UT ( #18566 )
2026-02-10 21:27:41 -08:00
Zheng Li and 瑀澈
27c447653d
model: support Qwen3.5 ( #18489 )
...
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com >
2026-02-10 00:27:59 +08:00
Xinyuan Tong
0b4d4f2838
Fix MMLU benchmark to auto-download data and resolve path issue ( #18486 )
2026-02-09 10:40:40 -05:00
Yuan Luo and luoyuan.luo
4ea4f2a20c
[VLM] Optimize get_rope_index for GLM4v ( #17420 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-02-01 18:59:15 +08:00
b8zhong
22498e10c0
[Fix] Triton TP MoE Dpsk V3/Qwen3 Coder with SwapAB ( #17965 )
2026-01-31 15:56:26 +08:00
Yuan Luo and luoyuan.luo
7bb41989fa
[1/N] Optimize All Reduce - Benchmark different AR operations ( #13797 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-01-26 22:44:13 +08:00
Jacob Gordon
a296c99ff4
refactor(benchmark): prevents variable shadowing ( #17607 )
2026-01-22 17:00:11 -05:00
Julian Huang and 墨楼
db2425a00b
[Fix]: correctly fetch ds32 config in tuning_fused_moe_triton ( #17409 )
...
Co-authored-by: 墨楼 <huangzhilin.hzl@antgroup.com >
2026-01-20 20:08:28 +08:00
Mohammad Miadh Angkad
b0701f02b3
Fix benchmark import for should_use_tensor_core ( #17232 )
2026-01-16 17:48:36 -05:00
Yongfei Xu
82a1b645ba
[DeepSeek V3.1/V3.2] Optimize fused moe configs for H20 & H20-3E based on swapab ( #17133 )
2026-01-17 00:10:52 +08:00
b8zhong
d44f09ad98
[Benchmark] Add GSM8K Platinum Eval ( #14565 )
2026-01-16 11:06:14 +08:00
JinYan Su and Zhiqiang Xie
72e2f70ef7
feat(hicache): support numa detect to reduce long tail latency ( #11028 )
...
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu >
2026-01-15 14:11:49 -08:00
elvischenv
1d811094f8
[Misc] Auto download question file for benchmark/mtbench ( #17019 )
2026-01-13 10:34:29 -05:00
Bhavneek Singh
559ff9ecaf
Bug: fixed multi_chain_reasoning test ( #16192 )
2026-01-12 09:06:41 -08:00
Yuan Luo and luoyuan.luo
d1ec93e3ac
Optimize layernorm_gated for Qwen3-Next ( #16397 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-01-10 20:55:31 +08:00
fzyzcjy
249c356331
Super tiny update tokenizer benchmark ( #16429 )
2026-01-05 09:14:52 +08:00
fzyzcjy
387fad2f74
Tiny add detokenization benchmarks ( #16400 )
2026-01-04 22:53:38 +08:00
roikoren755
b021332339
[NemotronH] Add latent MoE support ( #16227 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
2026-01-02 22:08:58 +08:00
satyamk7054 and Satyam Kumar
38dd4fbb66
Add overlap scheduling for embeddings code path ( #14032 )
...
Co-authored-by: Satyam Kumar <satyamk@linkedin.com >
2025-12-24 18:24:18 -08:00
Lee Nau
7e027691c8
update benchmark README to use --fp8-gemm-backend instead of env var ( #15689 )
2025-12-23 23:23:31 -08:00
Yubo Wang and Liangsheng Yin
762846531f
Fix Illegal Memory Access when fa3 + spec + topk + page_size > 1 ( #15469 )
...
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
2025-12-24 00:13:57 +08:00
Frank and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
9749d3e346
Update benchmarks to use HF token from environment. ( #15421 )
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-12-18 13:47:27 -08:00
zhangheng and Zhiqiang Xie
ea7c69ce28
[hotfix]: Add missing args for 3FS bench_client.py ( #14791 )
...
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu >
2025-12-14 11:13:19 -10:00
sglang-bot and sglang-bot
5c8bd8b51b
chore: bump SGLang version to 0.5.6.post2 ( #14858 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2025-12-11 12:29:52 -08:00
sglang-bot
9a327bdfcf
chore: bump SGLang version to 0.5.6.post1 ( #14651 )
2025-12-09 00:35:28 +08:00
Xiaoyu Zhang
03b835e7d1
Refactor tuning block wise kernel and opt Qwen/Qwen3-VL-32B-Instruct-FP8 ( #14141 )
2025-12-08 09:24:58 +08:00
laoyao0822
leavelet