This website requires JavaScript.
9c8777c80f
[AMD][Qwen3.5] aiter a8w8 gemm configuration (#19826 )
2026-03-13 15:23:58 +09:00
70d4aabe42
Add CLI args to conveniently support tuning more models (#12922 )
2026-03-13 14:10:55 +08:00
63ecdcbb18
Expose async LoRA interface to Offline Engine (#18636 )
2026-03-13 02:09:47 -04:00
d4e68ead1d
[quant] Ignore FP8 quantization layers (#20340 )
2026-03-13 13:59:39 +08:00
e00328d1e5
[Diffusion] Opt qwen-image-edit with fuse_residual_layernorm_scale_shift_gate_select01_kernel (#20395 )
2026-03-13 13:15:22 +08:00
197f807134
[RadixTree][7/N Refactor]: Refactor mamba radix tree, release dup kvcache in insert func (#19429 )
2026-03-13 12:28:32 +08:00
f605612b87
[HTTP] Fix /GET HTTP route when ollama endpoint is not set. (#20494 )
2026-03-12 20:54:32 -07:00
154af9e46c
update CI_PERMISSIONS.json (#20405 )
2026-03-13 10:50:35 +08:00
3deaf61345
misc: update CODEOWNERS (#20488 )
2026-03-12 19:36:17 -07:00
7ecf07b8f4
[jit_kernel] Temporarily Skip Flaky JIT Kernel GDN Test and Add PR Label (#20436 )
2026-03-13 09:34:22 +08:00
65dd08153d
Fix Test* mixin classes being collected as standalone pytest tests (#20417 )
2026-03-13 12:18:45 +11:00
9865f11421
[bugfix] fix bug when enable prefill delay and DP (#20134 )
2026-03-13 08:51:09 +08:00
c21ddbc785
[Minor] fix type annotations and invalid method calls in constrained … (#20132 )
2026-03-13 07:42:46 +08:00
78a467c74a
[AMD] [diffusion] feat: enable AITer GroupNorm for VAE decode on ROCm (#20170 )
2026-03-13 02:38:19 +03:00
f5a4a5429f
Revert early HTTP port reservation (#17754 , #19805 ) (#20468 )
2026-03-12 16:17:33 -07:00
9c6f166600
tiny fix the usage of test_logprobs (#20473 )
2026-03-12 15:37:45 -07:00
b227e53ebf
feat: add banner to sgl-model-gateway (#20471 )
2026-03-12 14:25:27 -07:00
af2807e146
[LoRA][I] Add MOE LoRA JIT alignment kernel and tests (#19710 )
2026-03-12 12:23:46 -07:00
e29305c120
[GDN] Add benchmark for sglang gdn prefill (#20428 )
2026-03-12 22:25:02 +08:00
a57a44739f
[diffusion] deps: upgrade diffusers from 0.36.0 to 0.37.0 (#20318 )
2026-03-12 19:17:28 +08:00
318a40fdfb
[Bug-fix] Fix gpu fault when run the test with dp-attention-enabled and max-concurrency is over 256 (#20399 )
2026-03-12 17:32:03 +08:00
4e5ca92249
[diffusion]: clear file-path-only outputs on all ranks to prevent TP GPU memory skew (#20353 )
2026-03-12 14:59:09 +05:30
5f1bfb0d28
[Security] Fix /rerun-ut bypassing run-ci gate for fork PRs (#20424 )
2026-03-12 02:24:29 -07:00
1e2983c98e
[AMD] Fix FP8 assertion failure in aiter MLA decode by falling back to self.k_scale (#19935 )
2026-03-12 16:48:51 +08:00
067353f67b
[Test] Refactor KL divergence and prefix cache branching to kits (#19715 )
2026-03-12 10:11:59 +02:00
46b558445d
Fix default_max_tokens compute error in responses api when mtp is opened (#18932 )
2026-03-12 16:00:48 +08:00
ac1310b300
Fix wrong BaseKVCacheMethod import in test_modelopt_fp8kvcache.py (#20403 )
2026-03-12 18:23:46 +11:00
2c03a5c6c7
Fix global server args not set error in test_triton_moe_wna16.py (#20412 )
2026-03-12 18:22:59 +11:00
91f701e5f1
[AMD][Mori] Add unittest for mori ep low latency and tbo (#20374 )
2026-03-12 13:38:26 +08:00
dd82678b2d
[NPU] Support mamba cache transfer for NPU (#20364 )
2026-03-12 12:49:21 +08:00
abc672e717
[Benchmark] use flashinfer bench_gpu_time instead of triton do_bench (#20305 )
2026-03-11 21:04:30 -07:00
ae7c2397b9
Fix FA3 swa spec pg_size > 1 (#20369 )
2026-03-12 11:42:01 +08:00
649d6f2bc8
[GDN] Change Attention State Layout from [N, HV, K, V] to [N, HV, V, K] (#20283 )
2026-03-12 10:53:12 +08:00
8787cf4566
Fix the scope of io_backend in NSATokenToKVPoolHost (#20327 )
2026-03-12 10:33:11 +08:00
7771fcd4b3
Update CodeOwners (#20329 )
2026-03-11 19:11:09 -07:00
9b55a98a67
perf(qwen3_5): replace einops rearrange with torch.flatten in GatedDe… (#20386 )
2026-03-11 18:51:27 -07:00
25bd83033d
Enable Piecewise CUDA Graph for NemotronH Hybrid (Mamba+Attention) Models (#19903 )
2026-03-11 18:16:38 -07:00
677e446e51
[NPU] Convert cu_window_seqlens to CPU for npu_flush_attention_unpad operator (#20328 )
2026-03-12 09:08:43 +08:00
67f02681c9
[AMD] Support speculative decoding v2 for aiter backend on ROCm/HIP (#17450 )
2026-03-11 17:01:01 -07:00
acab24a76a
fix: gracefully abort last request in retract_decode on OOM (#19881 )
2026-03-12 06:13:03 +08:00
88d2fc19b1
feature: support X-Data-Parallel-Rank header to specific dp-rank. (#19832 )
2026-03-12 05:53:33 +08:00
af4c28904d
[PD] Fix the infinite loop in deocde resolve_pending_reqs (#20371 )
2026-03-12 05:11:19 +08:00
ab4b863546
fix ci by removing nvidia-cutlass-dsl-libs-base and force reinstall n… (#20380 )
2026-03-11 13:37:33 -07:00
2720ea2667
[Typo] Fix H200 doc links pointing to H20 section in deepseek_v3.md (#20383 )
2026-03-11 13:35:20 -07:00
252ef90fc2
[Generative Score API] Fix on prefill-only scheduler running batch loss track problem (#14320 )
2026-03-11 13:15:50 -07:00
a54d71e967
[Benchmark] Add sglang-embedding backend to bench_serving (#20017 )
2026-03-11 13:13:16 -07:00
61b228239e
bump sgl-fa4 version to 4.0.5 to loose torch deps (#20378 )
2026-03-11 13:08:09 -07:00
006bd44cf9
[deepseekv3.2] fix get_k_and_s_triton kenel for 128K seqlen case bug (#19319 )
2026-03-12 03:56:33 +08:00
e6a6cd1f0c
[Fix] Reset output_ids for requests with input_embeds during retraction (#14110 )
2026-03-12 03:42:21 +08:00
7b44bc923e
Relax flaky B200 GSM8K accuracy thresholds (#20304 )
2026-03-11 12:35:21 -07:00
d093e70067
[Doc] Add DSA/NSA attention backend to support matrix (#20326 )
2026-03-11 10:40:35 -07:00
dae5c6cadf
[diffusion] doc: add Moore Threads as a supported vendor (#20146 )
2026-03-12 01:15:15 +08:00
05e40922b3
[Doc] Fix wrong link and cmd description (#20365 )
2026-03-11 23:30:45 +08:00
6cfc21ba74
[NPU] Bump SGL-Kernel-NPU version to 2026.03.10.rc1 (#20362 )
2026-03-11 22:28:21 +08:00
ed42af99a9
[NPU] [Quantization] w4a4 MoE layer support (#18924 )
2026-03-11 16:52:35 +03:00
9991debde3
[Feature] Integrate Elastic NIXL-EP into SGLang (#19248 )
2026-03-11 11:37:43 +02:00
680d9d98e4
Fix cutedsl ci error (#20309 )
2026-03-11 16:17:35 +08:00
456934fed5
feat: fix update last_receive_tstamp logic for health-check in multi-token-worker mode (#20256 )
2026-03-11 15:23:22 +08:00
61cad15d28
[Utils] Add NetworkAddress abstraction for IPv6-safe address handling (#20306 )
2026-03-11 00:07:37 -07:00
55e6acf834
[NPU][QwenVL] Support qwen image preprocess on npu (#20189 )
2026-03-11 15:03:08 +08:00
57b093dc34
[NPU]MindSpore backend support eagle3 (#17098 )
2026-03-11 14:11:19 +08:00
18cfeabd33
Add SGLANG_SORT_WEIGHT_FILES env var for sequential I/O optimization (#20194 )
2026-03-11 14:10:53 +08:00
4011065c65
Update CODEOWNERS (#20320 )
2026-03-11 14:08:07 +08:00
8c8a487468
[diffusion] doc: add diffusion-optimal-perf (#20311 )
2026-03-11 12:20:09 +08:00
c8bbe5010a
[diffusion] feat: add AITER Sage attention backend (#20178 )
2026-03-11 06:17:45 +02:00
21a0015aa3
[PCG]add piecewise cuda graph support for marlin linear (#20119 )
2026-03-11 10:57:08 +08:00
fe294904c9
[README] Add GB300 NVL72 inference blog post to News section (#20313 )
2026-03-10 19:31:37 -07:00
b2dd104ade
[Intel GPU] Upgrade pytorch xpu version to 2.10 (#20254 )
2026-03-11 07:17:25 +05:30
16ec4f3a4a
Integrate the AddRmsNorm operator (#19939 )
2026-03-11 09:05:04 +08:00
50953aea8d
[Scheduler] Unify idle checks into is_fully_idle() and fix weight update test (#20296 )
2026-03-10 17:50:23 -07:00
7a1ca53805
[AMD] Fix JSON mode streaming test flake on AMD CI (#20293 )
2026-03-10 17:24:11 -07:00
dc4380e33a
[AMD] [DeepSeek-OCR-2 Day 0] Enable DeepSeek-OCR-2 on AMD GPUs and add nightly test (#19732 )
2026-03-10 17:04:35 -07:00
eea7f8de41
[CI] Fix B200 runner label for scheduled runs (#20297 )
2026-03-10 16:00:31 -07:00
f32882cee5
[CI] Relax Eagle infer_b spec accept length threshold (#20300 )
2026-03-10 15:40:52 -07:00
09a118fafe
Support return_logprob for spec v2 (overlap safe) (#19801 )
2026-03-10 15:38:27 -07:00
76ee4bb98c
[FlashInfer v0.6.4] [RL] Integrate FlashInfer mxfp8 gemm, MoE, and routed MoE (#19537 )
2026-03-10 15:37:57 -07:00
bd460e9565
add logprob related params in bench_serving (#20218 )
2026-03-10 15:04:57 -07:00
db97f193b7
[diffusion][llm] macOS support (#19549 )
2026-03-11 04:11:07 +08:00
a3d88a247b
Enable piecewise-cuda-graph when logprob_start_len = -1 (#19453 )
2026-03-10 12:50:57 -07:00
031d0a2aad
[Qwen-MOE] Fix memory duplication issues in case layers weights are re-assigned during weight loading (#18255 )
2026-03-10 18:34:56 +01:00
11d9c36c2f
Replace soundfile+torchaudio with torchcodec AudioDecoder in load_audio (#20190 )
2026-03-10 17:26:29 +00:00
e1f0b3181a
[diffusion] fix: adjust convert_hf_to_fp8 to be compatible with more dits (#20281 )
2026-03-11 01:21:54 +08:00
60cc06297e
[4/n jit_kernel restruct] speed up CI tests and add benchmark workflow (#20268 )
2026-03-10 21:37:41 +08:00
5a7c1b8ec6
[NPU] replace swiglu with custom kernel
2026-03-10 21:08:37 +08:00
9884957c07
[NPU] Bugfix for qwen35 on NPU (#19756 )
2026-03-10 20:03:26 +08:00
6ed996bf65
[bugfix] disable share input buffer feature on npu due to accuracy issue (#19507 )
2026-03-10 19:26:46 +08:00
51d9d34977
[2/n jit_kernel restruct] unify rotary embedding entrypoints under rope.py (#20247 )
2026-03-10 17:49:57 +08:00
6407891b4f
[AMD] Fp8 prefill integration with radix cache path for dpsk models (#20187 )
2026-03-10 17:49:47 +08:00
751c454099
Add DeepSeek3.2 and GlmMoeDsa into moe tune (#18876 )
2026-03-10 17:12:58 +08:00
ac07a6d439
Revert "[Scheduler] Decouple maybe_send_health_check_signal from process_batch_result" (#20259 )
2026-03-10 01:58:48 -07:00
8cd1de3354
[diffusion] fix: map each prompt to corresponding image in multi-prompt scenario (#20081 )
2026-03-10 16:58:21 +08:00
2c2003158f
[diffusion] fix: fix flux2 lora (#20200 )
2026-03-10 16:57:01 +08:00
8517da5d08
[3/n jit_kernel restruct] Clean up benchmark naming and benchmarking helpers (#20250 )
2026-03-10 16:39:03 +08:00
c812504b92
[1/n jit_kernel restruct] unify cache usage and clean up naming in ngram_embedding (#20244 )
2026-03-10 15:53:43 +08:00
7cf0551014
Migrate norm kernels to FlashInfer JIT implementation (#18871 )
2026-03-09 23:56:07 -07:00
69158e9d9f
[Bugfix] Skip _mamba_verify_update for idle batch (#20167 )
2026-03-10 14:53:01 +08:00
834795adb8
[CI] Refactor PCG related CI (#19994 )
2026-03-09 23:49:25 -07:00
9b2e5526fb
[NPU][Bug fix] context parallel bug fix (#19820 )
2026-03-10 14:43:49 +08:00
5f717913a0
support Kimi-K2.5-w4a8 on ascend
2026-03-10 14:43:27 +08:00
dadd4dde83
[AMD] Skip the flaky test for lora ci test. (#20175 )
2026-03-10 14:15:30 +08:00
seungrokj