chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Commit Graph

  • 0b4d4f2838 Fix MMLU benchmark to auto-download data and resolve path issue (#18486) Xinyuan Tong 2026-02-09 10:40:40 -05:00
  • 006da22268 Pass quantize_config to _initialize_model (#18273) Kurt Shuster 2026-02-09 07:34:42 -08:00
  • ddbcfbaaab feature: support bidirectional attention for Gemma-3 (#10707) brimon 2026-02-09 23:17:45 +08:00
  • 4f7da5ad0f [diffusion] chore: fix unclean shutdown and resource leaks (#18477) Mick 2026-02-09 22:32:08 +08:00
  • 76eb1c8406 [diffusion] feat: add ModelScope support (#17924) yrk111222 2026-02-09 19:23:45 +08:00
  • 615a02dcd4 Revert "optimize get_topk_ragged by fusing get k and k_scale triton kernel" (#18471) Baizhou Zhang 2026-02-09 16:37:19 +08:00
  • 875ad6cf35 Tiny rename for spec related fileds. (#18468) Liangsheng Yin 2026-02-09 00:10:39 -08:00
  • 107958a489 Make compressed-tensors MoEs support ignored layers (#17828) LHXuuu 2026-02-09 14:37:33 +08:00
  • 14652243bd [DLLM] Add JointThreshold algorithm for joint M2T and T2T decoding (#18171) Junlin Zhou 2026-02-09 14:20:45 +08:00
  • 28717e3d28 [AMD] CI - Fix AMD daily image release and install dependency (#18452) YC Tseng 2026-02-09 14:20:09 +08:00
  • 3f3c201243 [AMD] Update aiter to v0.1.10.post2 (#18423) Bingxu Chen 2026-02-09 14:08:24 +08:00
  • b564dcec61 fix: use --no-build-isolation for human-eval install (#18455) Hudson Xing 2026-02-08 21:48:03 -08:00
  • a1189068fa fix: fix the wrong return value type of draft model runner (#18105) Yingchun Lai 2026-02-09 12:51:35 +08:00
  • 68e31a3485 [BugFix][PD]Fix metadata_buffer_index leak when aborted in PD (#17483) Zheng Wengang 2026-02-09 11:34:29 +08:00
  • bffd765417 Refactoring Mooncake TE as a shared distributed component (#17810) Shangming Cai 2026-02-09 10:53:11 +08:00
  • bf89cc3803 [ModelOPT] Support Qwen 3 Next Coder NVFP4 (#18224) Yi Zhong 2026-02-08 17:29:07 -05:00
  • 75997ebe8d Update author information in pyproject.toml (#18453) Lianmin Zheng 2026-02-08 12:22:55 -08:00
  • 071bf2ce09 [Kimi-K2.5] Fix missing quant_config in KimiK25 (#18440) Mohammad Miadh Angkad 2026-02-09 04:02:45 +08:00
  • 656a3d742e Add tensor parallelism support to LFM2 ShortConv layers (#17777) Piotr Mazurek 2026-02-08 17:52:47 +01:00
  • 6601bc24da [diffusion] chore: revise process title (#18446) Mick 2026-02-09 00:14:06 +08:00
  • 031a652b93 Fix TRT-LLM MLA backend applying k_scale to BF16 KV cache in BMM1 (#18396) debo3 2026-02-08 07:11:16 -08:00
  • a41aff1243 [diffusion] refactor: group component loaders under the component_loaders/ directory (#18438) Mick 2026-02-08 23:02:27 +08:00
  • ca36d88fa6 [ModelOpt] Fix broken Qwen3-235B-A22B-Instruct-2507-NVFP4 launch (#18189) Yi Zhong 2026-02-08 09:35:28 -05:00
  • 43eecd8265 [diffusion] feat: support efficient sequence shard (#18161) wxy 2026-02-08 21:09:39 +08:00
  • f600965b0e [CI] fix: notebook ci may not working (#18417) shuwenn 2026-02-08 14:26:35 +08:00
  • d71ccd8860 fix: sync server_args.kv_cache_dtype when detecting FP8 KV cache (#18394) Zack Yu 2026-02-07 22:10:59 -08:00
  • 8e2e835c2f [Fix] Fix backend selection after flashinfer version update (#18364) DarkSharpness 2026-02-08 11:20:41 +08:00
  • 00248d85c7 [diffusion] platform: support WAN/FLUX/Qwen-Image/Qwen-Image-edit on Ascend (#13662) Makcum888e 2026-02-08 05:45:30 +03:00
  • 7b83659310 fix: fix NVFP4 Kimi-K2.5 weight mapping and exclude list (#18370) Mohammad Miadh Angkad 2026-02-08 10:23:48 +08:00
  • 64950d8f97 [diffusion] feat: support saving videos directly on the server to avoid the overhead of tensor transfer (#18253) wxy 2026-02-07 22:08:42 +08:00
  • 31d4cd2ffd [diffusion] fix: respect dist_timeout option (#18386) Mick 2026-02-07 20:56:04 +08:00
  • fddef76619 [Doc] Fix outdated --fp4-gemm-backend documentation (#18350) Mohammad Miadh Angkad 2026-02-07 20:42:47 +08:00
  • d792aa7618 [diffusion] fix: remove unnecessary norm_type argument from GLM-Image dits (#18382) Hao Jin 2026-02-07 04:35:12 -08:00
  • eb4cf1dfc4 [CI] Skip some flaky subtests for test_multi_lora_backend.py (#18408) Baizhou Zhang 2026-02-07 19:06:53 +08:00
  • c47c2f9466 [Doc] Update CUDA 13 install guide to install torch first (#18404) Mohammad Miadh Angkad 2026-02-07 18:04:37 +08:00
  • 52401bec1d chore: bump mooncake version to 0.3.9 (#18316) Shangming Cai 2026-02-07 17:30:01 +08:00
  • baec650462 [Diffusion] Apply fused_norm_scale_shift to LTX2/MOVA (#18257) Xiaoyu Zhang 2026-02-07 17:28:42 +08:00
  • e834b85ab6 [NPU] update npu doc (#18344) Hexq0210 2026-02-07 16:38:05 +08:00
  • 1552aab741 Support execute_shell_command for env var support (#18390) 赵晨阳 2026-02-06 20:33:29 -08:00
  • 4637970dfb [Qwen3Next] Optimize fused_sigmoid_gating_delta_rule_update_kernel (#18271) hlu1 2026-02-06 19:59:42 -08:00
  • 9fbec79906 Revert "[Build] Enable full kernel in aarch64 wheel" (#18385) Baizhou Zhang 2026-02-07 09:19:07 +08:00
  • bedade1ef0 Merge stage-c-test-large-4-gpu suites into partitioned suites (#18325) Alison Shao 2026-02-06 15:32:33 -08:00
  • f1ff697494 add hybrid model PD to NIXL connector (#16229) Neal Vaidya 2026-02-06 15:05:36 -08:00
  • c850a8a41a [Docs] Add Falcon H1, Hunyuan-Large, Qwen3-Omni support and update Diffusion usage (#17888) Rishit Shivam 2026-02-07 02:47:51 +05:30
  • f2e0048d06 Add CI permission for Shunkangz, dongjiyingdjy, samuellees (#18377) Baizhou Zhang 2026-02-07 01:19:02 +08:00
  • e13b727e92 [diffusion] CI: update perf baseline (#17512) Prozac614 2026-02-07 00:28:44 +08:00
  • c6aa1863be Add Nemotron 3 Nano tests (#18119) shaharmor98 2026-02-06 17:55:42 +02:00
  • 79d409f210 [diffusion] fix: offload text encoder model in image encoding stage (#18317) xiaoye 2026-02-06 22:55:56 +08:00
  • 3d68bd9d9b add hicache jit test (#17847) Xuchun Shang 2026-02-06 16:54:33 +08:00
  • f798ab9775 [diffusion] fix: fix torch.compile graph break caused by torch._dynamo.disable (#18336) 陈一涵 2026-02-06 14:48:09 +08:00
  • d0c39bc219 Fix cross-container HF download race condition in CI (#18328) Alison Shao 2026-02-05 21:01:41 -08:00
  • 92b8bd6833 fix npu best practice (#18330) amote-i 2026-02-06 10:14:46 +08:00
  • aa390d2762 [Kernel] Migrate GPTQ-Marlin GEMM kernel to JIT (#18067) Linyu Wu 2026-02-06 08:31:42 +08:00
  • ef1d0ea885 [Doc] add a summary section for spec decode document (#18323) shuwenn 2026-02-06 05:34:31 +08:00
  • 8b21dd4b77 [Doc] refine spec decode docs for SpecV2/STANDALONE/NGRAM (#18321) shuwenn 2026-02-06 04:12:33 +08:00
  • 6a4b81e2d9 Refactor(qwen3-vl) optimize position encoding interpolation (#16781) aaaandychen 2026-02-06 02:26:35 +08:00
  • d22163eb8c Fix flaky test_frequency_penalty_reduces_word_repetition by using deterministic seeds (#18285) Alison Shao 2026-02-05 10:24:18 -08:00
  • 498d8d0680 NixlKVManager optimizations (#17654) ovidiusm 2026-02-05 17:25:23 +01:00
  • b639779dd8 [diffusion] feat: allow T5's TP Group to reuse the transformer's SP Group (#17818) wxy 2026-02-06 00:12:19 +08:00
  • 3f32a5831d throw error if got adapter with added_tokens (#18046) Glen Liu 2026-02-05 10:55:43 -05:00
  • 2eb4359ada [Kernel] Add JIT apply_rope_with_cos_sin_cache_inplace (#18155) pansicheng 2026-02-05 21:49:37 +08:00
  • 4aa03d91fd [diffusion] fix: fix accuracy bug caused by #14717 (#18296) 陈一涵 2026-02-05 20:36:18 +08:00
  • 8f8c1724ae docker: add patch to increase GPU deepep timeout (#18298) ishandhanani 2026-02-05 04:26:15 -06:00
  • afae4c7178 [PD] Minor code cleanup for mooncake backend (#18279) Shangming Cai 2026-02-05 17:38:09 +08:00
  • 079fc8f3c5 [piecewise graph]: support MiniMax-M2 (#18217) zhangheng 2026-02-05 15:24:38 +08:00
  • 3f1df322f9 [FIX] Always support TP > 4 for FP4 Gemm (#17300) danielafrimi 2026-02-05 09:10:26 +02:00
  • 368936a62b [XPU] Integrate MoE and minor improvements in XPU attention backend (#13561) Meng, Hengyu 2026-02-05 15:09:59 +08:00
  • dff3ba202a [Diffusion] Support layerwise offload for mova (#18272) Xiaoyu Zhang 2026-02-05 13:16:07 +08:00
  • c910829708 Fix test_return_routed_experts to use response-level sglext (#18274) Alison Shao 2026-02-04 20:16:01 -08:00
  • e616d35847 Support Markdown/Notebook-Friendly Documentation Export for Downstream Integration(convert rat files to md files and save) (#18278) Kun Lin 2026-02-04 22:59:40 -05:00
  • de6a03260f [docs] fix misspellings & typos (#18276) rinbaro 2026-02-04 19:35:29 -08:00
  • c8212b9fac [PD] doc: Document SGLANG_MOONCAKE_CUSTOM_MEM_POOL and supported values (#18259) Teng Ma 2026-02-05 11:03:33 +08:00
  • f730c18679 [PD] improve kv offset calculation for MHA model with different tp size (#18163) Ch3ngY1 2026-02-05 10:43:23 +08:00
  • f218234e4f [diffusion] chore: prohibit Chinese characters usage (#18249) Mick 2026-02-05 09:22:26 +08:00
  • 599c5f4922 fix kimi k2.5's moe gemm config init (#18064) yinghui 2026-02-05 07:59:01 +07:00
  • bdaf3de9b3 fix: add SGLANG_IS_IN_CI env var to release-docs workflow (#18225) zwang86 2026-02-04 15:49:41 -08:00
  • c1d5cc3b24 [Bugfix] fix a obvious logic error (#18254) linhaifeng 2026-02-05 05:59:58 +08:00
  • efbf39583e Add MoE fused config for Qwen3-Coder-Next-FP8 on H100 TP=2 (#18195) Mohammad Miadh Angkad 2026-02-05 05:36:35 +08:00
  • 2e87c2bd5e fix: fix MockModelRunner in attention tests (#18240) Zack Yu 2026-02-04 13:18:02 -08:00
  • 6fd878b41d [AMD] Add kimi mi35x nightly test, folder organization and several stability fixes (#17895) Michael 2026-02-04 12:03:57 -08:00
  • 36a3e78af9 [diffusion] refactor: move model_stages into stages folder (#18248) Mick 2026-02-05 00:23:31 +08:00
  • 3e7ecb78a6 model: support interns1-pro (#18145) RunningLeon 2026-02-05 00:22:44 +08:00
  • a6f53cc5e3 entrypoint: support passing spaces_between_special_tokens per request (#17939) RunningLeon 2026-02-04 22:18:36 +08:00
  • 4c403045ec [diffusion] fix: fix the bug of redundant memory usage on GPU-0 (#18221) wxy 2026-02-04 21:25:23 +08:00
  • 0c9a0adc53 [diffusion] chore: clean MOVA codes (#18107) Zhang Yiyang (SII) 2026-02-04 21:23:41 +08:00
  • 760ae933bb optimize get_topk_ragged by fusing get k and k_scale triton kernel (#16043) BingjiaWang 2026-02-04 19:59:41 +08:00
  • 315306d8a9 Make sure we always disable symm memory without dp padding (#18129) Nicolas Castet 2026-02-04 05:58:28 -06:00
  • a72f4f839c Tiny fix for fp8 moe backend flashinfer_trtllm naming (#18243) Jincong Chen 2026-02-04 19:58:04 +08:00
  • ce02df8592 [diffusion] logging: downgrade default prompt log from info to debug (#17813) Evrard-Nil 2026-02-04 12:19:02 +01:00
  • 2e9d0442e2 [diffusion] update code owner (#18247) Xiaoyu Zhang 2026-02-04 19:12:32 +08:00
  • 84c09913eb Moving _alloc_extend_naive out of npu allocator (#18200) Cheng Wan 2026-02-04 02:09:55 -08:00
  • be557cbc5f [RadixTree][5/N Refactor]: Introduce pre and post-processing methods for key matching (#18147) zhangheng 2026-02-04 17:10:46 +08:00
  • d279520ba5 [DeepGemm] Add a flag for fast warmup (#18111) Baizhou Zhang 2026-02-04 14:12:13 +08:00
  • 4739f2e8d5 [diffusion] kernel: gated residual layernorm scale shift and layernorm scale shift kernel fusion for Qwen-Image, WAN and HunyuanVideo (#14717) Jianying 2026-02-04 13:46:20 +08:00
  • 669a9bd180 Support Markdown/Notebook-Friendly Documentation Export for Downstream Integration (copy all markdown and rst files) (#18223) Kun Lin 2026-02-03 23:53:17 -05:00
  • b7c1dfc602 fix: bumping nightly whl version (#18212) Douglas Yang 2026-02-03 20:43:38 -08:00
  • 37c33cc0aa fuse qkvbfg linear into one gemm and f_b g_b into batched gemm. (#17801) strgrb 2026-02-04 11:41:26 +08:00
  • c1d529c196 Fix Session for multimodal and expose it through Engine (#18152) Aurick Qiao 2026-02-03 18:33:27 -08:00
  • 1f72f66c6d [Docs] fix readme typo (#18207) Qi Jia 2026-02-04 09:37:28 +08:00
  • da758ed601 [diffusion] fix: fix server cache-dit bug under continuous dynamic requests (#17140) wxy 2026-02-04 09:03:37 +08:00