chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,198 Commits 1 Branch 0 Tags
45adad37d0f59cd5a6b3cb0ecc2ee40f93ac8b66
Commit Graph

8198 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
fzyzcjy
45adad37d0 Add manual routing policy for router (#15586) 2025-12-25 11:27:36 +08:00
fzyzcjy
1ba897f330 Tiny refactor select_workers API for future passing more information (#15596) 2025-12-25 10:31:10 +08:00
satyamk7054
38dd4fbb66 Add overlap scheduling for embeddings code path (#14032) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-12-24 18:24:18 -08:00
Lianmin Zheng
ecd2d09a6c Fix code sync scripts (#15787) 2025-12-24 16:13:23 -08:00
Lianmin Zheng
92ddc46824 [Auto Sync] Update server_args.py (20251223) (#15700) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-12-24 14:55:14 -08:00
Simo Lin
5454d2a7ae [model-gateway] release smg 0.3.0 (#15781) 2025-12-24 13:23:57 -08:00
Chenxi Li
17b38f88bc Add LoRA metrics for potential auto scaling (#15149) 2025-12-24 12:13:52 -08:00
Xinyuan Tong
ae434f7821 Move limit-mm-data-per-request to make code clean (#15775) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-24 10:59:39 -08:00
Simo Lin
643aeefe6d [model-gateway] Fix logging module name, parse endpoint context, and tokenizer factory (#15782) 2025-12-24 10:54:48 -08:00
ゆり
186a56f6e2 fix(monitoring): update Grafana dashboard metrics prefix from sglang: to sglang_ (#15758) 2025-12-24 10:43:51 -08:00
Douglas Yang
17e65466de fix: nightly fix b200 gpqa (#15745) 2025-12-24 10:29:45 -08:00
Praneth Paruchuri
370bd27f3a [model-gateway] Implement Zero-Copy Vision Tensor Access (#15750) 2025-12-24 09:15:44 -08:00
Simo Lin
b27b5a834e [bug] fix code formatting which blocks ci (#15780) 2025-12-24 09:14:13 -08:00
Simo Lin
2f7c629235 [model-gateway] Fix IGW routing and optimize RouterManager (#15741) 2025-12-24 11:28:53 -05:00
Mayyyy
2fb3160598 [NPU] Bug fix in device detect (#14137) 2025-12-25 00:18:39 +08:00
Liwansi
8bf7f240b6 [NPU]qwen3 pp bugfix (#15390) 2025-12-25 00:14:40 +08:00
Mick
2c5679f314 [diffusion] refactor: unify the profiling api for all executors (#15718) 2025-12-24 23:26:09 +08:00
Liangsheng Yin
159b128357 Tiny add flush for CI crash locating (#15769) 2025-12-24 22:47:10 +08:00
fzyzcjy
9338f63f7d Add request counter in addition to existing response counter (#15768) 2025-12-24 22:22:38 +08:00
fzyzcjy
8196998a03 Tiny add num retracted tokens metric (#15653) 2025-12-24 22:15:23 +08:00
fzyzcjy
aa21c6e3f1 Super tiny code cleanup (#15652) 2025-12-24 22:15:00 +08:00
fzyzcjy
fd4a558e71 Add metrics for having prefill and decode in different ranks (#15752) 2025-12-24 21:35:35 +08:00
fzyzcjy
b3b818fd86 Tiny unify realtime_tokens_total metric (#15747) 2025-12-24 21:29:43 +08:00
Liangsheng Yin
d5fbbfd988 Retry removing wrong logic about max total token in spec decoding (#15748) 2025-12-24 21:13:26 +08:00
fzyzcjy
e245cac0c5 Support JSON format request logging for easier parsing (#15743) 2025-12-24 19:52:43 +08:00
fzyzcjy
c6a6ba4368 Tiny refactor request logger (#15740) 2025-12-24 19:51:26 +08:00
fzyzcjy
d6108166bc Fix smg_http_requests_total semantics (#15655) 2025-12-24 17:41:01 +08:00
vincentzed
ddb3970eb4 Tiny env cleanup in deepgemm (#15706) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-12-24 00:58:46 -08:00
Arthur Cheng
f65fa04748 [model-gateway]Enable IGW mode with gRPC router and auto enable IGW when service discovery is turned on (#15459) 2025-12-24 00:27:58 -08:00
Lee Nau
7e027691c8 update benchmark README to use --fp8-gemm-backend instead of env var (#15689) 2025-12-23 23:23:31 -08:00
Liangsheng Yin
cb719c74ad [2/N] clean duplicate code of logprob processing in spec. (#15593) 2025-12-24 15:21:13 +08:00
Lianmin Zheng
ff903a7eea Simplify server args (#15704) 2025-12-23 22:46:12 -08:00
Prozac614
eee3700d84 [diffusion] feat: support lora strength (#15691) 2025-12-24 13:31:23 +08:00
Yuwei An
e254cdf326 [CI] Remove pcg-omni-ci (#15656) 2025-12-24 13:28:05 +08:00
Mick
dfb5357448 [diffusion] http-server: relax openai image endpoint's strict content_type limit (#15717) 2025-12-24 13:26:13 +08:00
Simo Lin
9665574937 [docs] major SGL Model Gateway documentation update (#15715) 2025-12-23 20:26:09 -08:00
vincentzed
ac320a6f04 Move some quant args to its own section in environ variables doc (#15722) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-12-23 20:11:08 -08:00
Liangsheng Yin
6292c24437 Tiny fix test eagle infer b. (#15716) 2025-12-24 11:57:15 +08:00
Liangsheng Yin
5f5a567768 Tiny add flush in the suite partition status print. (#15719) 2025-12-24 11:55:59 +08:00
Xuchun Shang
3bf07c684f [Feature][MM] split the images of one request into multiparts (#11828) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Kun(llfl) <llfl@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
 2025-12-24 11:22:04 +08:00
michael-amd
e7b09efc0a [AMD] Add AMD Nightly Performance & VLMs Accuracy Tests (#15500) 2025-12-23 19:03:27 -08:00
fzyzcjy
99d3bcdfed [model-gateway] add back router worker health metric and fix init state (#15622) 2025-12-23 18:50:38 -08:00
fzyzcjy
4d64f15086 [mode;-gateway] add back fixes of incorrect metrics after worker removal (#15624) 2025-12-23 18:50:03 -08:00
Qiaolin Yu
aef7ca7cf2 Raise the accept length bar in dpsk-r1-fp4 spec decoding tests (#15705) 2025-12-23 18:38:02 -08:00
Xuchun Shang
fe712aa3df [bug fix] fix hicache jit kernel (#15177) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-24 10:31:21 +08:00
Simo Lin
846953d9f1 [model-gateway] Add tokenize/detokenize HTTP endpoints and tokenizer management (#15702) 2025-12-23 17:32:07 -08:00
Roger Young
5c64a20da7 Update MiniMax-M2 ToolCall and add MiniMax-M2.1 in Docs (#15538) 
Co-authored-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-12-23 15:11:52 -08:00
Lianmin Zheng
cf81737693 Add kv_transfer_total_mb to metrics (#15667) 2025-12-23 11:16:55 -08:00
Simo Lin
aa6ac96635 [model-gateway] Fix tokenizer caching and improve error handling (#15695) 2025-12-23 10:33:38 -08:00
Liangsheng Yin
0d0367e9d0 Tiny fix CI (#15696) 2025-12-24 01:51:08 +08:00