chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,520 Commits 1 Branch 0 Tags
42fcf5438ff94e97ff691fe6e2d8144a0a87e895
Commit Graph

133 Commits

Author SHA1 Message Date
Baizhou Zhang
42fcf5438f Revert "tiny remove deprecated endpoint call" (#14533) 2025-12-05 23:48:54 -08:00
b8zhong
ec7b2c16d9 tiny remove deprecated endpoint call (#13607) 2025-12-05 09:54:49 -08:00
Xiaoyu Zhang
9c80072845 Add peak output tokens per second in bench_serving (#14165) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-01 17:47:54 +08:00
fzyzcjy
487c8d4df3 Tiny add several args to bench serving (#14181) 2025-12-01 14:45:47 +08:00
Binyao Jiang
90c18a16cb [GLM4.6v] Required changes for bumping up to transformer 5.x (#13229) 2025-11-18 10:58:00 +08:00
Douglas Yang
7ee3e36412 Fix: test_vlm_offline_throughput output throughput (#13279) 2025-11-15 14:31:48 +08:00
Ling Zhang
0050ff254f [BugFix] fix bench_serving error when multimodal image is testing (#13254) 2025-11-14 22:46:19 +08:00
Rohan Potdar
c9b581644d Dump total_throughput to output-file in bench_serving.py (#9790) 2025-11-12 18:48:28 +08:00
Ziming Huang
e8114102fa [Fix] Update text_chunks in bench_serving chat completions (#13041) 2025-11-11 11:41:11 -08:00
elvischenv
71aea45c41 [Fix] Add TPOT back to bench_serving (#12976) 2025-11-11 17:32:24 +08:00
yinghui
b88fab3111 fix: add seed bench_serving to cache key, remove redundant function definition. (#12680) 2025-11-04 23:39:11 -08:00
Glen Liu
cbf23dbbfa [Feature] add --lora-request-distribution arg to bench_serving.py and support skewed and distinct workloads (#12175) 2025-11-04 21:41:40 -08:00
fzyzcjy
8834260739 Super tiny dump server info such as args in bench for post analysis (#12550) 2025-11-03 14:24:08 -08:00
fzyzcjy
fd7a72d62d Super tiny allow profile activities in bench_serving (#12549) 2025-11-03 14:23:18 -08:00
Yi Zhang
21a8fa16ea tiny optimize for bench serving (#12553) 2025-11-03 14:13:18 -08:00
Ke Bao
f7f9e41b36 Fix run benchmark (#12473) 2025-11-01 02:39:48 +08:00
daniel, chen
cf0c24150a add served model name in bench serving (#12428) 2025-11-01 01:28:11 +08:00
Zaili Wang
50b6842b4b fix: Add default value for backend in sample_mmmu_requests (#12256) 2025-10-31 19:31:40 +08:00
fzyzcjy
fb52d35f63 Super tiny fix AMD ci (#12378) 2025-10-29 23:25:18 -07:00
fzyzcjy
25c5049870 Super tiny add tag for benchmark scripts (#12340) 2025-10-30 11:19:14 +08:00
Liangsheng Yin
ea385ae85a Fix ITL metrics when using openai endpoint with spec (#12156) 2025-10-26 18:06:25 +08:00
Xinyuan Tong
b9fb74f3bc fix: bench_serving ITL calculation when using spec-decoding (#12064) 2025-10-24 17:02:44 +08:00
Zaili Wang
92009bd28e fix: fix MMMU loading issue (#11759) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-23 20:21:38 -07:00
b8zhong
d0a64c7e2c vlm: enforce pybase64 for image and str encode/decode (#10700) 2025-10-21 19:05:32 +08:00
Xiaoyu Zhang
88a6f9dab5 bench_serving support PD Disaggregation (#11542) 2025-10-13 19:43:26 -07:00
Mick
0c0779d667 ci: improve nightly-ci (#11385) 2025-10-12 21:19:34 -07:00
shaharmor98
fa7e2c3049 fix bench_serving mishandling of internal states (#11376) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
 2025-10-09 19:24:50 +08:00
Kevin Xiang Li
e3bb7f5ae6 benchmark: enhance configurable multimodal benchmarking in bench_serving (#9812) 
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-10-08 01:31:36 -07:00
Lifu Huang
92473e2e34 Support LoRA in bench_serving oai interface (#11318) 2025-10-08 01:28:58 -07:00
fzyzcjy
fdc4e1e570 Tiny move files to utils folder (#11166) 2025-10-03 22:40:06 +08:00
Muqi Li
51f7c6bd3c Add auth to get server info (#10751) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-27 02:54:39 -07:00
Muqi Li
bd95944cf6 [Bugfix][Minor][Benchmark] Fix some bugs due to PR #10495 (#10982) 2025-09-26 22:01:05 -07:00
Mick
777eb53897 ci: refactor nightly test (#10495) 2025-09-26 15:24:30 -07:00
Zhihao Zhang
e7bc600304 [Feature] Speculative decoding support lookahead (#9873) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-18 16:42:41 -07:00
Muqi Li
d5e2a37414 Benchmark: Support API_KEY without 'bearer' (#10380) 2025-09-12 16:29:04 -07:00
blzheng
97fff98c68 [CPU] Fix phi4-mm prompt issue in bench_serving (#9900) 2025-09-08 20:12:32 -07:00
Yineng Zhang
19d64f2b72 fix: resolve lint issue (#10181) 2025-09-08 15:09:55 -07:00
Teng Ma
a02071a12c [Bench] feat: mooncake trace integration (#9839) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
 2025-09-09 02:50:54 +08:00
Mick
16a6d21b95 chore: enhance bench_serving for vlms with a new dataset of configurable image count and resolution (#9583) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
 2025-08-26 17:42:54 -07:00
Mick
584e1ab2d0 fix: fix unsupported palette mode of images in bench_serving for mmmu (#9206) 2025-08-14 18:44:46 -07:00
Brayden Zhong
a37e1247c1 [Multimodal][Perf] Use pybase64 instead of base64 (#7724) 2025-07-08 14:00:58 -07:00
Lianmin Zheng
22352d47a9 Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-06-29 23:16:19 -07:00
Xinyuan Tong
c45e49d817 oai: Adds support for OpenAI chat completions API in bench_serving (#7036) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-06-28 22:59:20 +00:00
Zijian
31d6dee5c4 Support VILA models (#6106) 2025-06-11 11:47:25 -07:00
Xinyuan Tong
697b0f71f0 [Refactor] image data process in bench_serving (#6879) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-06 21:11:17 -07:00
Yikai Zhang
fb507b7b10 [FIX] mmmu bench serving result display error (#6525) (#6791) 2025-05-31 13:48:06 -07:00
Yuhong Guo
d279d4990c Fix aiohttp 'Chunk too big' in bench_serving (#6737) 2025-05-30 00:50:36 -07:00
fzyzcjy
501efc3d36 Tiny fix CI (#6611) 2025-05-25 23:36:34 -07:00
fzyzcjy
6bebef60a7 Support accurate length control for bench serving (#6594) 2025-05-25 22:46:23 -07:00
fzyzcjy
25be63d0b2 Auto handle PD disaggregation in bench_serving (#6587) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-05-25 22:41:27 -07:00