chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • 38c233fd04 [Piecewise] Support PCG weak_ref_tensor cuda kernel on AMD (#17291) Binyao Jiang 2026-01-20 14:05:32 -08:00
  • 20ed3822bb [Auto Sync] Update piecewise_cuda_graph_runner.py (20260119) (#17313) Lianmin Zheng 2026-01-20 14:05:05 -08:00
  • 4ecd9afde9 [Docs] Rename SGLang Router to SGLang Model Gateway (#17436) zijiexia 2026-01-20 12:31:10 -08:00
  • eb38d64413 [Docs] Explain CUDA attention backend choices and aiter FP8 KV cache support (#17428) Ruihuan He 2026-01-20 14:10:36 -05:00
  • 6ea491e439 Overlap shared experts with deepep dispatch for single batch overlap on Blackwell (#17289) Baizhou Zhang 2026-01-21 02:56:55 +08:00
  • 16802fb6b2 [FIX] fix mambaish model pp kv cache compute (#17334) GuoweiWangU 2026-01-20 23:59:05 +08:00
  • d97066d209 Use attn_tp_group for all reduce in token embedding (#17403) Ke Bao 2026-01-20 23:56:21 +08:00
  • ce2d686e94 Disable swa memory for trtllm-mha backend (#17429) Ke Bao 2026-01-20 23:55:00 +08:00
  • 76b06bee03 [New Model] GLM4.7-Flash (#17247) Qiaolin Yu 2026-01-20 07:44:16 -08:00
  • 612026ad2c [OPT] DeepSeekV3.2: optimize indexer weight_proj-mma performance (#17205) BingjiaWang 2026-01-20 23:09:44 +08:00
  • 55c616427d Add flag that enables NCCL mlp sync batch for overlap scheduler (#17288) Baizhou Zhang 2026-01-20 23:06:55 +08:00
  • 91a4cd8648 Update default attn backend to fa3 for mimo model (#17419) Ke Bao 2026-01-20 21:22:27 +08:00
  • 23d765d1f2 [Doc] Update pipeline parallelism documentation (#17414) Shangming Cai 2026-01-20 20:10:56 +08:00
  • db2425a00b [Fix]: correctly fetch ds32 config in tuning_fused_moe_triton (#17409) Julian Huang 2026-01-20 20:08:28 +08:00
  • 603f386c6b update docs of Ascend plateform (#17358) amote-i 2026-01-20 19:31:23 +08:00
  • f7a5e425c3 [3/N] Achieve fault tolerance at the DP level (#11657) ympcMark 2026-01-20 18:47:08 +08:00
  • d50dcd9b61 fix: Release pr pypi fix (#17382) Douglas Yang 2026-01-19 23:00:54 -08:00
  • e6b7c04947 [Kimi-Linear] Refactor kimi-linear gate calculation to avoid duplicated code (#17160) Yuan Luo 2026-01-20 14:29:24 +08:00
  • a3addd6203 [AMD] Add DeepSeek-V3.2 and VLMs model in nightly tests (#17179) Michael 2026-01-19 20:31:56 -08:00
  • 6988a0f570 Disable mla persistent kernel when not using fp8 kv_cache (#17327) Thomas Wang 2026-01-20 12:29:54 +08:00
  • 1b192cf198 [NPU] Update NPU doc for model and features supported (#17385) Hexq0210 2026-01-20 12:28:06 +08:00
  • c560e14216 [diffusion] fix: enforce 16-pixel alignment for flux2 to prevent shape mismatch crash (#17302) Ratish P 2026-01-20 09:33:33 +05:30
  • 71cb9d0302 [PD] Fix PP dynamic chunking with DP attention (#17339) Shangming Cai 2026-01-20 11:27:13 +08:00
  • 8fb45523f3 feat: support bitsandbytes quantization algorithm (#15325) shuwenn 2026-01-20 10:36:56 +08:00
  • 84aef37808 fix function calling for Trinity (#17364) Raghav Ravishankar 2026-01-20 08:03:17 +05:30
  • 17c04b109d Enable parallel stage execution for scheduled CI runs (#16880) Alison Shao 2026-01-19 18:31:46 -08:00
  • 55c4288b3e Fix runner utilization workflow to use 24h default (#17378) Alison Shao 2026-01-19 18:31:33 -08:00
  • 9f8b79f16f [Docs] Fix formatting in Evaluating New Models with SGLang (#17376) zijiexia 2026-01-19 18:22:30 -08:00
  • 7dc3cbe7ca [Docker] Fix CUDA 13 installing wrong nvidia-nccl-cu13 due to nixl-cu13 not breaking system package (#17370) b8zhong 2026-01-19 17:52:30 -08:00
  • 79ddc34c1c [Docs] Add new model evaluation docs (#17043) zijiexia 2026-01-19 16:35:03 -08:00
  • 09a9d214f7 Pipe customized_info through CudaGraphRunner output (#17088) Aurick Qiao 2026-01-19 15:48:49 -08:00
  • 7e40d52635 Move test_autoround.py to stage-b-test-large-1-gpu suite (#17336) Alison Shao 2026-01-19 14:36:19 -08:00
  • 057b07fc50 Disable unit-test-backend-4-gpu-gb200 job (#17367) Alison Shao 2026-01-19 14:35:58 -08:00
  • 91d8c52d6d fix: updating pypi workflow with new base version formatting in pyproject.toml (#17366) Douglas Yang 2026-01-19 13:23:11 -08:00
  • e9a44ea607 [AMD] fix perf ci errors (#17363) YC Tseng 2026-01-20 05:04:15 +08:00
  • c1282da236 fix(ci): apply MMMU retry logic to all affected test files (#17329) Hudson Xing 2026-01-20 02:00:29 +08:00
  • 71279e31f7 [CI] fix test_vlm_models.py (#17049) shuwenn 2026-01-20 02:00:00 +08:00
  • 1a053a810c [AMD] CI - add partitions for stage-b-test-small-1-gpu-amd (#17345) YC Tseng 2026-01-20 00:16:07 +08:00
  • 2ea02f0642 [AMD CI] Migrate and Add More Testcases (#17116) Bingxu Chen 2026-01-20 00:07:39 +08:00
  • 20b0523eca [RadixTree][1/N Refactor]: Support unified match_prefix params (#17142) zhangheng 2026-01-19 22:39:40 +08:00
  • ce8a6ac690 Evict swa kv cache during decoding (#17220) Ke Bao 2026-01-19 22:36:52 +08:00
  • ebca5879a1 Fix v32 continue_final_message not work (#16567) ybyang 2026-01-19 21:45:39 +08:00
  • cc410a1088 [Diffusion] Apply qknorm to flux2 and apply lightx2v rms_norm_one_pass kernel(without residual) (#17305) Xiaoyu Zhang 2026-01-19 21:25:33 +08:00
  • f374623fa9 [Refactor] Set fp4-gemm-backend=auto on SM100 and rename fp4-gemm-backend with flashinfer_ prefix (#17309) b8zhong 2026-01-19 04:09:07 -08:00
  • 5c02217746 Inclusion of nvfp4 blockscale in EPLB Rebalance (#17158) Shu Wang 2026-01-19 03:45:27 -06:00
  • 8916b9d080 Migrate performance, accuracy, and quantization tests to CI registry (#17177) Alison Shao 2026-01-18 23:25:24 -08:00
  • a3d9a21882 Revert "[Perf] fuse q, k norm for Flux2Attention (#17241)" (#17332) Xiaoyu Zhang 2026-01-19 15:24:11 +08:00
  • fb88fb672e fix(ci): rate limit and permission errors in trace publishing (#17238) Alison Shao 2026-01-18 23:20:22 -08:00
  • 2d72e168fd [CI] Add partition to stage-b-test-large-1-gpu (11->12) (#17245) Alison Shao 2026-01-18 23:19:39 -08:00
  • 64946679b5 [Perf] fuse q, k norm for Flux2Attention (#17241) Minglei Zhu 2026-01-18 22:33:34 -08:00
  • 5836324c55 KV Cache Events with Attention DP bug fix (#16030) (#16412) Kartik Ramesh 2026-01-19 00:13:48 -06:00
  • 9fe56cd0fb Fix kernel selection in biased_grouped_topk_gpu (#17325) yudian0504 2026-01-19 14:06:11 +08:00
  • 858a4d659b support new qwen3_coder_detector (#16744) Gaoji Liu 2026-01-19 13:16:27 +08:00
  • e619f53113 [Auto Sync] Update tokenizer_manager.py (20260119) (#17317) Lianmin Zheng 2026-01-18 20:32:19 -08:00
  • fc4b932f4e Update code sync scripts (#17319) Lianmin Zheng 2026-01-18 19:57:28 -08:00
  • d2105d4abd [DeepSeek v3.2] Opt MTP decode cuda batch sizes and nsa implementation (#16961) Yongfei Xu 2026-01-19 11:54:11 +08:00
  • 84c8390514 Use dsv3 optimized routing fused_topk_deepseek instead of moe_fused_gate (#15347) Lee Nau 2026-01-18 19:50:16 -08:00
  • ea879c7739 [Minor] Correct sglang version when installing from source (#17315) Baizhou Zhang 2026-01-19 11:36:16 +08:00
  • 0227db8926 [PD] Optimize MHA models pp util calculation logic (#17306) Shangming Cai 2026-01-19 11:23:15 +08:00
  • ad1b4e4728 [Feature] overlap LoRA weight loading with compute (#15512) Glen Liu 2026-01-18 21:43:17 -05:00
  • 51f147ada3 Update CODEOWNERS for multimodal_gen (#17308) Mick 2026-01-19 08:39:20 +08:00
  • d3eafc7357 [GLM 4.7] Add RTX 6000 Pro aka sm120 (#17235) Koushik Dutta 2026-01-18 13:33:19 -08:00
  • e00b43442d [jit-kernel] Add CuTe DSL GDN Decode Kernel (#15631) Jinyan Chen 2026-01-19 04:54:36 +08:00
  • 733de6be31 [NPU]Support GPT-OSS for NPU (#14197) Todobe 2026-01-19 04:13:41 +08:00
  • 93433726eb [Refactor] Split out deepseek v2 weight loader function into mixin (#16649) Jerry Ji 2026-01-18 08:41:59 -08:00
  • 330605cc88 [Diffusion] Apply jit qk_norm to flux1 (#17296) Xiaoyu Zhang 2026-01-19 00:28:36 +08:00
  • bb6055b43c [Bugfix] Fix PD accuracy when MTP is not configured on the prefill node (#17212) Ch3ngY1 2026-01-18 23:37:58 +08:00
  • 4df74eb576 [Refactor] Add -fp4-gemm-backend to replace SGLANG_FLASHINFER_FP4_GEMM_BACKEND (#16534) b8zhong 2026-01-18 07:25:46 -08:00
  • f3a7c7dcd9 Move radix cache related tests (#17295) Ke Bao 2026-01-18 17:47:32 +08:00
  • 2069050d3f fix: Handle multiple named chat templates in HuggingFace tokenizers (#17236) Xinyuan Tong 2026-01-18 09:20:04 +00:00
  • 1fe0c82f67 Add kl test for swa radix cache (#17281) Ke Bao 2026-01-18 16:19:57 +08:00
  • a45e0e5df4 [SPEC_V2] Enable cudagraph draft_extend for trtllm_mla_backend and Acclen Fix for DP under cudagraph mode (#16974) YAMY 2026-01-17 23:56:21 -08:00
  • f78201f3a9 Tiny fix comment typo (#17287) Ke Bao 2026-01-18 15:06:33 +08:00
  • 8fd3399880 [diffusion] fix: set guidance_scale default to None (#17182) Changyi Yang 2026-01-18 02:04:11 -05:00
  • 088758c1c1 [Tiny] Improve docs (#17264) Mohammad Miadh Angkad 2026-01-18 14:57:01 +08:00
  • 6d29d8ab16 [VLM][Reland] Refactor load_mm_data to improve performance (#16152) Yuan Luo 2026-01-18 14:11:17 +08:00
  • e499258e97 Use swa radix cache and memory pool for gpt-oss model (#17261) Ke Bao 2026-01-18 13:47:07 +08:00
  • 09491a9bcd cli: support sglang version (#17250) Mick 2026-01-18 13:20:24 +08:00
  • 7edb06158e Add runner utilization report workflow (#17234) Alison Shao 2026-01-17 19:28:05 -08:00
  • e486a4dac1 [diffusion] feat: support default 4-step inference for Flux2-Klein distilled models (#17225) Lancer 2026-01-18 10:15:50 +08:00
  • 90399cbc07 fix(ci): recover from corrupted MMMU parquet cache (#17256) Hudson Xing 2026-01-18 09:32:01 +08:00
  • 53609e5e5b Revert "[Diffusion] Move diffusion time embedding to jit kernel" (#17257) Michael 2026-01-17 05:29:22 -08:00
  • 9c2530642c Change routing policy API to be async to support more policies (#17048) fzyzcjy 2026-01-17 17:12:51 +08:00
  • d2c863878c [DLLM] Implement initial dynamic batching for diffusion LLM (#14883) Zehuan Li 2026-01-17 16:48:15 +08:00
  • 737a1183d6 [BUGFIX] fix radix cache memory consumption to avoid OOM (#17191) Yi Zhang 2026-01-17 16:47:37 +08:00
  • dc743fe4ba [diffusion] chore: clean srt imports (#17252) Mick 2026-01-17 15:47:49 +08:00
  • dd99f818e0 fix: fix regression and unclear pattern (#16561) Nan Jiang 2026-01-17 15:21:42 +08:00
  • 8ce64aa155 fix(ci): skip offline mode for LoRA scenarios (#17248) Hudson Xing 2026-01-17 15:06:20 +08:00
  • eb768189ee [diffusion] doc: add instruction for adding performance baseline of new model (#17249) Mick 2026-01-17 12:24:59 +08:00
  • 2cdd4370bc [Diffusion] Move diffusion time embedding to jit kernel (#16879) Xiaoyu Zhang 2026-01-17 12:21:22 +08:00
  • a7b5f75d88 Support integration tests with Redis binary (#17045) fzyzcjy 2026-01-17 11:59:04 +08:00
  • 305c1a575a Tiny remove unused code (#17047) fzyzcjy 2026-01-17 11:58:43 +08:00
  • c824ddd581 Fix imbalancedness for manual policy min group mode for requests without routing id (#17044) fzyzcjy 2026-01-17 11:58:35 +08:00
  • e18e0057a4 Add an env var to allow transferring small metadata via TCP for PD (#16951) siyu 2026-01-17 10:32:22 +08:00
  • 166396ca4c [Docs] minor update on ep docs (#17242) zijiexia 2026-01-16 18:20:47 -08:00
  • 43779f27b7 Disable PCG for draft worker (#16354) Stefan He 2026-01-16 17:56:56 -08:00
  • 8b9e9357fe [2/n] deepseek_v2.py Refactor: Migrate MHA forward method in deepseek_v2.py (#16817) Baizhou Zhang 2026-01-17 09:36:25 +08:00
  • d36f6f043c [Fix] flashinfer_trtllm intermediate_size assertion with Qwen3 + TP=8 (#16824) b8zhong 2026-01-16 17:24:05 -08:00
  • b0701f02b3 Fix benchmark import for should_use_tensor_core (#17232) Mohammad Miadh Angkad 2026-01-17 06:48:36 +08:00
  • 4229de3b13 [model-gateway] Refine TokenizerRegisty.load() to handle duplication (#17230) Chang Su 2026-01-16 13:53:21 -08:00