 Praneth Paruchuri
|
370bd27f3a
|
[model-gateway] Implement Zero-Copy Vision Tensor Access (#15750)
|
2025-12-24 09:15:44 -08:00 |
|
|
Simo Lin
|
b27b5a834e
|
[bug] fix code formatting which blocks ci (#15780)
|
2025-12-24 09:14:13 -08:00 |
|
|
Simo Lin
|
2f7c629235
|
[model-gateway] Fix IGW routing and optimize RouterManager (#15741)
|
2025-12-24 11:28:53 -05:00 |
|
|
Mayyyy
|
2fb3160598
|
[NPU] Bug fix in device detect (#14137)
|
2025-12-25 00:18:39 +08:00 |
|
|
Liwansi
|
8bf7f240b6
|
[NPU]qwen3 pp bugfix (#15390)
|
2025-12-25 00:14:40 +08:00 |
|
|
Mick
|
2c5679f314
|
[diffusion] refactor: unify the profiling api for all executors (#15718)
|
2025-12-24 23:26:09 +08:00 |
|
|
Liangsheng Yin
|
159b128357
|
Tiny add flush for CI crash locating (#15769)
|
2025-12-24 22:47:10 +08:00 |
|
|
fzyzcjy
|
9338f63f7d
|
Add request counter in addition to existing response counter (#15768)
|
2025-12-24 22:22:38 +08:00 |
|
|
fzyzcjy
|
8196998a03
|
Tiny add num retracted tokens metric (#15653)
|
2025-12-24 22:15:23 +08:00 |
|
|
fzyzcjy
|
aa21c6e3f1
|
Super tiny code cleanup (#15652)
|
2025-12-24 22:15:00 +08:00 |
|
|
fzyzcjy
|
fd4a558e71
|
Add metrics for having prefill and decode in different ranks (#15752)
|
2025-12-24 21:35:35 +08:00 |
|
|
fzyzcjy
|
b3b818fd86
|
Tiny unify realtime_tokens_total metric (#15747)
|
2025-12-24 21:29:43 +08:00 |
|
|
Liangsheng Yin
|
d5fbbfd988
|
Retry removing wrong logic about max total token in spec decoding (#15748)
|
2025-12-24 21:13:26 +08:00 |
|
|
fzyzcjy
|
e245cac0c5
|
Support JSON format request logging for easier parsing (#15743)
|
2025-12-24 19:52:43 +08:00 |
|
|
fzyzcjy
|
c6a6ba4368
|
Tiny refactor request logger (#15740)
|
2025-12-24 19:51:26 +08:00 |
|
|
fzyzcjy
|
d6108166bc
|
Fix smg_http_requests_total semantics (#15655)
|
2025-12-24 17:41:01 +08:00 |
|
|
vincentzed
|
ddb3970eb4
|
Tiny env cleanup in deepgemm (#15706)
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
|
2025-12-24 00:58:46 -08:00 |
|
|
Arthur Cheng
|
f65fa04748
|
[model-gateway]Enable IGW mode with gRPC router and auto enable IGW when service discovery is turned on (#15459)
|
2025-12-24 00:27:58 -08:00 |
|
|
Lee Nau
|
7e027691c8
|
update benchmark README to use --fp8-gemm-backend instead of env var (#15689)
|
2025-12-23 23:23:31 -08:00 |
|
|
Liangsheng Yin
|
cb719c74ad
|
[2/N] clean duplicate code of logprob processing in spec. (#15593)
|
2025-12-24 15:21:13 +08:00 |
|
|
Lianmin Zheng
|
ff903a7eea
|
Simplify server args (#15704)
|
2025-12-23 22:46:12 -08:00 |
|
|
Prozac614
|
eee3700d84
|
[diffusion] feat: support lora strength (#15691)
|
2025-12-24 13:31:23 +08:00 |
|
|
Yuwei An
|
e254cdf326
|
[CI] Remove pcg-omni-ci (#15656)
|
2025-12-24 13:28:05 +08:00 |
|
|
Mick
|
dfb5357448
|
[diffusion] http-server: relax openai image endpoint's strict content_type limit (#15717)
|
2025-12-24 13:26:13 +08:00 |
|
|
Simo Lin
|
9665574937
|
[docs] major SGL Model Gateway documentation update (#15715)
|
2025-12-23 20:26:09 -08:00 |
|
|
vincentzed
|
ac320a6f04
|
Move some quant args to its own section in environ variables doc (#15722)
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
|
2025-12-23 20:11:08 -08:00 |
|
|
Liangsheng Yin
|
6292c24437
|
Tiny fix test eagle infer b. (#15716)
|
2025-12-24 11:57:15 +08:00 |
|
|
Liangsheng Yin
|
5f5a567768
|
Tiny add flush in the suite partition status print. (#15719)
|
2025-12-24 11:55:59 +08:00 |
|
|
|
3bf07c684f
|
[Feature][MM] split the images of one request into multiparts (#11828)
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Kun(llfl) <llfl@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
|
2025-12-24 11:22:04 +08:00 |
|
|
michael-amd
|
e7b09efc0a
|
[AMD] Add AMD Nightly Performance & VLMs Accuracy Tests (#15500)
|
2025-12-23 19:03:27 -08:00 |
|
|
fzyzcjy
|
99d3bcdfed
|
[model-gateway] add back router worker health metric and fix init state (#15622)
|
2025-12-23 18:50:38 -08:00 |
|
|
fzyzcjy
|
4d64f15086
|
[mode;-gateway] add back fixes of incorrect metrics after worker removal (#15624)
|
2025-12-23 18:50:03 -08:00 |
|
|
Qiaolin Yu
|
aef7ca7cf2
|
Raise the accept length bar in dpsk-r1-fp4 spec decoding tests (#15705)
|
2025-12-23 18:38:02 -08:00 |
|
|
Xuchun Shang
|
fe712aa3df
|
[bug fix] fix hicache jit kernel (#15177)
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
|
2025-12-24 10:31:21 +08:00 |
|
|
Simo Lin
|
846953d9f1
|
[model-gateway] Add tokenize/detokenize HTTP endpoints and tokenizer management (#15702)
|
2025-12-23 17:32:07 -08:00 |
|
|
|
5c64a20da7
|
Update MiniMax-M2 ToolCall and add MiniMax-M2.1 in Docs (#15538)
Co-authored-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-12-23 15:11:52 -08:00 |
|
|
Lianmin Zheng
|
cf81737693
|
Add kv_transfer_total_mb to metrics (#15667)
|
2025-12-23 11:16:55 -08:00 |
|
|
Simo Lin
|
aa6ac96635
|
[model-gateway] Fix tokenizer caching and improve error handling (#15695)
|
2025-12-23 10:33:38 -08:00 |
|
|
Liangsheng Yin
|
0d0367e9d0
|
Tiny fix CI (#15696)
|
2025-12-24 01:51:08 +08:00 |
|
|
Liangsheng Yin
|
bd572360f3
|
Tiny apply gsm8k mixin to ngram test (#15606)
|
2025-12-24 01:30:26 +08:00 |
|
|
ratish
|
5f3a47d8a7
|
[model-gateway]: add gRPC router embeddings endpoint implementation (#15273)
|
2025-12-23 09:12:22 -08:00 |
|
|
Praneth Paruchuri
|
80ae2229d3
|
[model-gateway] Optimize router selection with lock-free snapshots (#15672)
|
2025-12-23 09:09:30 -08:00 |
|
|
Liangsheng Yin
|
705287b2e5
|
Tiny add more information in retract logging. (#15694)
|
2025-12-24 00:43:36 +08:00 |
|
|
Yubo WangandLiangsheng Yin
|
762846531f
|
Fix Illegal Memory Access when fa3 + spec + topk + page_size > 1 (#15469)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-12-24 00:13:57 +08:00 |
|
|
Arthur Cheng
|
dd620987d1
|
[model-gateway] Replace tokenizer with tokenizer registry for dynamic tokenizer loading in gRPC router (#12968)
|
2025-12-23 07:58:47 -08:00 |
|
|
shuwenn
|
53f974b973
|
fix: potential crash for missing stream attribute (#15644)
|
2025-12-23 23:24:31 +08:00 |
|
|
fzyzcjy
|
6a5764a719
|
Super tiny add test_soft_watchdog to nightly (#15692)
|
2025-12-23 22:46:15 +08:00 |
|
|
DarkSharpness
|
291f11ae39
|
[Minor] Enhance JIT kernel and add dev docs (#14570)
|
2025-12-23 22:34:59 +08:00 |
|
|
Teng MaandXuchun Shang
|
d7301c89ba
|
[Feature] support fastsafetensors (#15091)
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
|
2025-12-23 22:33:56 +08:00 |
|
|
shuwenn
|
758b9067a0
|
[CI] fix UT assert error in test_tokenizer_manager.py (#15646)
|
2025-12-23 22:24:33 +08:00 |
|