chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,796 Commits 1 Branch 0 Tags
d1ec93e3acc483d37e33b1378f2b2ff7efbb3cfc
Commit Graph

64 Commits

Author SHA1 Message Date
fzyzcjy
32a569fb77 Tiny add CPU resource monitoring for overload diagnosis (#16852) 2026-01-10 12:48:18 +08:00
fzyzcjy
3ed3b7ef7c Tiny add routing key distribution metrics (#16847) 2026-01-10 12:06:40 +08:00
fzyzcjy
6406a5969b Tiny support customizing prometheus buckets for prefill delayer (#16831) 2026-01-10 08:17:37 +08:00
fzyzcjy
cec19b56c6 Tiny add command line args for prefill delayer and unify names (#16830) 2026-01-10 07:54:25 +08:00
fzyzcjy
8eeffbe9aa Support token low usage watermark in prefill delayer (#16814) 2026-01-09 23:07:18 +08:00
fzyzcjy
7656d26737 Tiny enhance prefill delayer observability (#16812) 2026-01-09 22:51:16 +08:00
fzyzcjy
9a21d89c5b Tiny add metrics for prefill delayer (#16603) 2026-01-07 09:53:52 +08:00
fzyzcjy
9821fae5c6 Tiny add explanations to realtime token metric labels (#16009) 2026-01-03 12:36:55 +08:00
Chenxi Li
17b38f88bc Add LoRA metrics for potential auto scaling (#15149) 2025-12-24 12:13:52 -08:00
fzyzcjy
8196998a03 Tiny add num retracted tokens metric (#15653) 2025-12-24 22:15:23 +08:00
fzyzcjy
fd4a558e71 Add metrics for having prefill and decode in different ranks (#15752) 2025-12-24 21:35:35 +08:00
fzyzcjy
b3b818fd86 Tiny unify realtime_tokens_total metric (#15747) 2025-12-24 21:29:43 +08:00
Lianmin Zheng
cf81737693 Add kv_transfer_total_mb to metrics (#15667) 2025-12-23 11:16:55 -08:00
fzyzcjy
88a405cc10 Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401) 2025-12-18 22:24:23 +08:00
fzyzcjy
c5f4e20f2f Support GPU execution time breakdown by forward mode metrics (#15396) 2025-12-18 22:18:17 +08:00
Scott Lee
169a75dfb8 Add request-level timestamp for when prefill finishes (#14860) 2025-12-17 13:34:46 -08:00
fzyzcjy
3e690cce53 Add realtime token counter metrics (#15198) 2025-12-17 21:20:24 +08:00
fzyzcjy
b12b40dea4 Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189) 2025-12-17 21:19:42 +08:00
fzyzcjy
fdc93b019c Add sglang:decode_sum_seq_lens metric (#15066) 2025-12-14 08:42:45 +08:00
Lianmin Zheng
796d82b107 [Auto Sync] Add max_total_num_tokens metric: Update scheduler_metrics_mixin.py, collector.py (20251202) (#14256) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dan Zheng <dzheng@x.ai>
 2025-12-01 16:34:34 -08:00
Lianmin Zheng
a352e833c4 CI: Kill zombie diffusion processes in CI & minor code style fix on rotary embedding fallback (#13637) 2025-11-20 21:57:01 +08:00
Lianmin Zheng
17b24aca6e [Auto Sync] Update base_grammar_backend.py, collector.py (20251116) (#13357) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-11-19 16:05:50 -08:00
yinghui
38a704bccb refine stdout logging codes (#13015) 2025-11-12 00:16:14 +08:00
ShawnKung
afee2843d5 feat(metrics): add scheduler and hiradix cache metrics (#10218) (#10225) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-11-10 18:14:47 +08:00
yinghui
58095cb00a Add timing metrics for requests (#12646) 
Co-authored-by: Scott Lee <scottjlee@users.noreply.github.com>
 2025-11-05 23:07:16 -08:00
Lianmin Zheng
55dfb539cf [Auto Sync] Update scheduler_metrics_mixin.py, collector.py (20251104) (#12647) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-11-04 13:56:14 -08:00
Lianmin Zheng
f600866a44 Improve the metrics for PD (#12580) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-11-03 22:10:57 -08:00
Lianmin Zheng
6a63a9852e minor code sync (#12403) 2025-10-30 12:49:31 -07:00
Yingchun Lai
5e36a0b455 [metrics][EPLB]: Support selected count of physical experts on each GPU (#9825) 2025-10-28 20:56:19 -07:00
Scott Lee
bf8f7a944f Add per-request retraction count (#11177) 2025-10-27 23:22:34 -07:00
Chang Su
627974405d [Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685) 2025-10-17 16:49:46 -07:00
Lianmin Zheng
b9a54e0968 [minor] sync code on python/sglang/test/test_deterministic.py and improve ci tests (#11777) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-17 14:25:22 -07:00
Lianmin Zheng
b98cf39866 [Auto Sync] Update collector.py (20251014) (#11625) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-14 13:34:33 -07:00
Scott Lee
b6fb5d7666 Add metrics for speculative decoding (acceptance rate, average acceptance length) (#11441) 2025-10-13 11:24:27 -07:00
Scott Lee
55b14656e6 Revert "Add metrics for speculative decoding (acceptance rate, average acceptance length)" (#11433) 2025-10-10 12:54:57 -07:00
Scott Lee
0babd48736 Add metrics for speculative decoding (acceptance rate, average acceptance length) (#11144) 2025-10-10 00:46:44 -07:00
Lianmin Zheng
6a261aaca5 Minor fixes for server_args, parallel_state, and test_deterministic.py (#11159) 2025-10-02 12:12:49 -07:00
Lianmin Zheng
2d62af6be5 Fix metrics and request tracing (TimeStats) (#11123) 2025-10-01 13:03:07 -07:00
Lianmin Zheng
f68dd998b9 Rename customer label -> custom label (#10899) 
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-25 16:19:53 -07:00
Yingchun Lai
b1721edbac [PD metrics] Add latency Histogram metrics of each stage for generate requests (#8710) 2025-09-16 01:52:49 +08:00
Yingchun Lai
fc2c3a3d8e metrics: support customer labels specified in request header (#10143) 2025-09-14 20:00:08 -07:00
Yingchun Lai
21ca4c3afa [PD metrics] Fix some uncompleted PD related metrics (#8627) 2025-09-14 02:26:58 -07:00
Lianmin Zheng
a06bf66425 [Auto Sync] Update collector.py, startup_func_log_and_timer... (20250910) (#10242) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-09 18:05:16 -07:00
pansicheng
f84db115b1 Add storage read/write bandwidth logs to monitor kvcache performance (#9965) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-05 16:52:55 -07:00
Yingchun Lai
b32ab0705e metrics: support customer buckets for prompt/generation_tokens_histogram (#9634) 2025-09-04 22:22:08 +08:00
SCDESPERTATE
b5c6529e17 [PD] Improve disaggregation metrics output: update the metrics to keep reflecting real stats (#7317) 2025-08-24 23:16:43 -07:00
Zhao Chen
3586b4cef2 feat: add production metric for retracted requests due to insufficient kvcache (#7030) 
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
 2025-07-17 11:59:05 -07:00
Lianmin Zheng
20fd53b8f6 Correctly abort the failed grammar requests & Improve the handling of abort (#6803) 2025-06-01 19:00:07 -07:00
Lifu Huang
3cf1473a09 Use monotonic clock for interval measurement (#6211) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-17 16:49:18 -07:00
Lianmin Zheng
01bdbf7f80 Improve structured outputs: fix race condition, server crash, metrics and style (#6188) 2025-05-11 08:36:16 -07:00