chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • ef32677444 Fix positional argument (#7093) Quanfeng Li 2025-06-12 09:31:13 +08:00
  • 021f76e4f4 [Perf] Refactor LoRAManager to eliminate stream syncs and redundant computations (#6994) Lifu Huang 2025-06-11 16:18:57 -07:00
  • 777688b892 [feat]: Emit fixed-size KV blocks events (#6824) Faradawn Yang 2025-06-11 15:07:58 -05:00
  • 0ca594eda9 [FIX]remove redundant code in logits_processor.py (#7079) Neo 2025-06-12 02:49:30 +08:00
  • 31d6dee5c4 Support VILA models (#6106) Zijian 2025-06-12 02:47:25 +08:00
  • 02543b545c Fix misusing the "_is_cuda". (#7091) sogalin 2025-06-12 02:21:31 +08:00
  • 25a6a9aa22 Fix circular import in test_prefix_chunk_info.py (#7097) Baizhou Zhang 2025-06-11 10:57:45 -07:00
  • 83d87685c5 vlm: adapt internvl to VisionAttention (#6870) Mick 2025-06-11 16:16:04 +08:00
  • 2a5f0100e0 Fix GGuf and add back test_gguf.py (#7067) Baizhou Zhang 2025-06-10 21:07:20 -07:00
  • dbdf76ca98 Clean up docs for server args and sampling parameters (generated by grok) (#7076) Lianmin Zheng 2025-06-10 19:55:42 -07:00
  • f2a75a66c4 update doc (#7046) Ximingwang-09 2025-06-11 10:02:01 +08:00
  • 6b12d6a8d5 Simplify the heuristics for setting --mem-fraction-static (#7054) Lianmin Zheng 2025-06-10 19:01:39 -07:00
  • 0f218731e3 Do not run frontend_reasoning.ipynb to reduce the CI load (#7073) Lianmin Zheng 2025-06-10 17:15:31 -07:00
  • 14c18d25df Frontend language separate reasoning support (#6031) Yudi Xue 2025-06-10 17:11:29 -07:00
  • 90bd3e32d6 Improve perf tuning docs (#7071) Lianmin Zheng 2025-06-10 16:55:04 -07:00
  • ca9291181d [Feature] Add Logit Bias (#6579) Brayden Zhong 2025-06-10 18:39:25 -04:00
  • 344adb00ec fix arm sgl-kernel link issue (#7066) Yineng Zhang 2025-06-10 15:23:22 -07:00
  • b56de8f943 Open AI API hidden states (#6716) kyle-pena-kuzco 2025-06-10 17:37:29 -04:00
  • ce5ee3bdf0 chore: update dev docker (#7064) Yineng Zhang 2025-06-10 12:57:58 -07:00
  • a0e4d4eb53 Fix missing tool call id if tool call index >0 in streaming tool call output. (#7049) Xu Wenqing 2025-06-11 03:56:43 +08:00
  • 2f58445531 Revert "Add sanity checks when a test file is not added to CI (#6947)" (#7063) Yineng Zhang 2025-06-10 12:43:25 -07:00
  • fe55947acd Add sanity checks when a test file is not added to CI (#6947) fzyzcjy 2025-06-11 03:34:57 +08:00
  • 19995dd78e Tiny fix cutlass_mla_get_workspace_size stub incorrect signature (#7057) fzyzcjy 2025-06-11 03:27:57 +08:00
  • 3b014bc13d Fix test_lora.py CI (#7061) Baizhou Zhang 2025-06-10 12:24:46 -07:00
  • d7c3e8e93d [fix] libmlx5.so already in base image (#7060) Hank Han 2025-06-11 03:22:24 +08:00
  • 8ea7df6114 [WA] fix output data is nan in CI test "test_moe_eval_accuracy_large.py" (#7021) kk 2025-06-11 00:08:10 +08:00
  • 4a102a2b02 Minor style fix in cuda_graph_runner.py (#7053) Lianmin Zheng 2025-06-10 06:32:41 -07:00
  • 6406408a70 Clean up server_args.py (#7037) Lianmin Zheng 2025-06-10 05:34:29 -07:00
  • 019851d099 Fix eagle on AMD (#7051) Lianmin Zheng 2025-06-10 05:22:40 -07:00
  • 2dae104dca Minor cleanup of fa3 backend (#6999) Lianmin Zheng 2025-06-10 03:58:44 -07:00
  • cef6655b26 fix 24.12 docker (#7045) Yineng Zhang 2025-06-10 02:43:22 -07:00
  • 27196d4148 [Docker] Upgrading base image from 24.04 to 24.12 (#7043) Swipe4057 2025-06-10 12:23:05 +03:00
  • bb185b0e92 Update README.md (#7040) Lianmin Zheng 2025-06-10 01:59:14 -07:00
  • 4f723edd3b chore: bump v0.4.7 (#7038) Yineng Zhang 2025-06-10 01:56:20 -07:00
  • fcde67b016 CPU: map changes from developing branch in sgl-kernel (#6833) YanbingJiang 2025-06-10 16:08:15 +08:00
  • 81372f3bef Fix fused_moe triton configs (#7029) yudian0504 2025-06-10 14:23:03 +08:00
  • baa6624d7c [CI] Add CI workflow for sgl-router docker build (#7027) Arthur Cheng 2025-06-09 23:16:44 -07:00
  • c2b16795b5 Add decode req pool (#6980) Byron Hsu 2025-06-09 21:23:36 -07:00
  • f6ebba537a Support both approximate and exact expert distribution collection (#6964) fzyzcjy 2025-06-10 11:56:17 +08:00
  • 6716b41786 Update default settings for blackwell (#7023) Baizhou Zhang 2025-06-09 20:37:47 -07:00
  • 1c8b42c84c chore: update pr test xeon (#7018) Yineng Zhang 2025-06-09 17:36:25 -07:00
  • f20f70003d Fix torch version in blackwell dockerfile (#7017) Qiaolin Yu 2025-06-09 20:06:50 -04:00
  • f40942ad63 Migrate to assertEqual (#6741) Emmanuel Ferdman 2025-06-10 02:47:39 +03:00
  • dc0705a504 Simplify prepare_extend_after_decode (#6987) Lianmin Zheng 2025-06-09 16:39:21 -07:00
  • a968c888c0 Fix torchvision version for Blackwell (#7015) Wenxuan Tan 2025-06-09 17:50:19 -05:00
  • a979daac3b Fallback to lower triton version for unfound fused moe configs (#7013) Baizhou Zhang 2025-06-09 15:41:03 -07:00
  • f1569876d5 feat: add direct routing strategy to DP worker (#6884) ishandhanani 2025-06-09 11:44:05 -07:00
  • 3465d7ae78 Update amd nightly models CI. (#6992) Sai Enduri 2025-06-09 10:54:08 -07:00
  • e58423b2b9 Fix cutlass MLA gets almost zero accuracy (#6998) fzyzcjy 2025-06-10 01:16:29 +08:00
  • 7059ae16fb chore: update pr test xeon (#7008) Yineng Zhang 2025-06-09 10:08:44 -07:00
  • 51d9a597f9 cleanup tmp dir (#7007) Yineng Zhang 2025-06-09 09:26:04 -07:00
  • 56ccd3c22c chore: upgrade flashinfer v0.2.6.post1 jit (#6958) Yineng Zhang 2025-06-09 09:22:39 -07:00
  • 98c00a2df1 Fix torch profiler bugs for bench_offline_throughput.py (#6557) Yueyang Pan 2025-06-09 20:33:41 +08:00
  • 451ffe74d9 support qwen3 emebedding (#6990) Pan Lyu 2025-06-09 16:32:49 +08:00
  • b1e5a33ae3 Eliminate stream sync to speed up LoRA batch init (#6960) Lifu Huang 2025-06-09 00:22:45 -07:00
  • 9d5fa68b90 Use torch.compile to fuse flash attention decode metadata preparation (#6973) Lianmin Zheng 2025-06-08 23:05:40 -07:00
  • 2c18642502 Enable more unit tests for AMD CI. (#6983) Sai Enduri 2025-06-08 19:41:55 -07:00
  • 18efb5e8e0 [perf][sgl-kernel] extend cutlass_mla_decode to support num_head < 128 (#6929) JieXin Liang 2025-06-09 10:37:34 +08:00
  • de1350ea20 Minor remove one kernel for DeepSeek (#6977) fzyzcjy 2025-06-09 08:41:35 +08:00
  • 86fe943bc3 Fix expert distribution dumping causes OOM (#6967) fzyzcjy 2025-06-09 08:41:14 +08:00
  • 9ecb18568b Fix triton sliding window test case (#6981) Lianmin Zheng 2025-06-08 17:20:46 -07:00
  • cc74499d51 Fix draft extend ut stability with flush cache (#6979) Ke Bao 2025-06-09 08:09:32 +08:00
  • 0c1f03a23d Sync cuda graph runners (#6976) Lianmin Zheng 2025-06-08 16:12:25 -07:00
  • 3712abfaf9 Fuse routed scaling factor in deepseek (#6970) Xiaoyu Zhang 2025-06-09 06:24:24 +08:00
  • 971a0dfa32 Extend cuda graph capture bs for B200 (#6937) Baizhou Zhang 2025-06-08 05:13:22 -07:00
  • 2fc1299562 Remove unnecessary kernels of num_token_non_padded (#6965) fzyzcjy 2025-06-08 20:09:17 +08:00
  • 20d3ad3b58 Fix CI and triton moe Configs (#6974) Lianmin Zheng 2025-06-08 05:06:46 -07:00
  • fa3592cfeb rebase h20 fused_moe config (#6966) Xiaoyu Zhang 2025-06-08 20:01:34 +08:00
  • 608668e143 Slightly improve the sampler to skip unnecessary steps (#6956) Lianmin Zheng 2025-06-08 03:18:54 -07:00
  • 6c0a48282a chore: bump sgl-kernel v0.1.7 (#6963) Yineng Zhang 2025-06-08 02:43:15 -07:00
  • 4740288303 [AMD] Add more tests to per-commit-amd (#6926) Hubert Lu 2025-06-08 01:08:37 -07:00
  • 1fb76ebb93 Revert "Fuse routed scaling factor in topk_reduce kernel (#6220)" (#6968) Yineng Zhang 2025-06-07 21:02:49 -07:00
  • c2c4f57f63 [DeepseekR1-FP4] Add Support for nvidia/DeepSeekR1-FP4 model (#6853) Pavani Majety 2025-06-07 17:24:35 -07:00
  • 23881fa60c chore: upgrade sgl-kernel v0.1.6.post1 (#6957) Yineng Zhang 2025-06-07 17:18:55 -07:00
  • 8db3ac55a9 chore: bump sgl-kernel v0.1.6.post1 (#6955) Yineng Zhang 2025-06-07 15:25:46 -07:00
  • 3e56f557fd Add a CUDA kernel for fusing mapping and weighted sum for MoE. (#6916) Elfie Guo 2025-06-07 15:24:39 -07:00
  • 62fec60d81 Add H20 fused MoE kernel tuning configs for DeepSeek-R1/V3 (#6885) Xu Wenqing 2025-06-08 06:17:34 +08:00
  • e7759778e5 [misc] add is_cpu() (#6950) JieXin Liang 2025-06-08 06:13:45 +08:00
  • 77e928d00e Update server timeout time in AMD CI. (#6953) Sai Enduri 2025-06-07 15:10:27 -07:00
  • 515ef4facb Fuse routed scaling factor in topk_reduce kernel (#6220) Xiaoyu Zhang 2025-06-08 02:06:50 +08:00
  • f5599ef124 Refactor global_server_args_dict (#6866) fzyzcjy 2025-06-07 18:10:35 +08:00
  • c499591ac8 Add canary for EPLB rebalancing (#6895) fzyzcjy 2025-06-07 18:09:33 +08:00
  • e1ce44cdb1 Disabling mixed chunked prefill when eagle is enabled (#6874) Swipe4057 2025-06-07 13:06:58 +03:00
  • f1114e7ff3 [Docker] Add docker file for SGL Router (#6915) Arthur Cheng 2025-06-07 02:58:43 -07:00
  • bae4fdc7ab add fbgemm moe grouped gemm kernel benchmark (#6924) Xiaoyu Zhang 2025-06-07 17:57:30 +08:00
  • 6153f2ff6e chore: upgrade sgl-kernel v0.1.6 (#6945) JieXin Liang 2025-06-07 17:53:26 +08:00
  • 8b5f83ed3b reduce torch.zeros overhead in moe align block size kernel (#6369) Xiaoyu Zhang 2025-06-07 17:47:36 +08:00
  • 2a413829f4 Add triton version as a fused_moe_triton config search key to avoid performace decrease in different Triton version (#5955) Xiaoyu Zhang 2025-06-07 17:43:50 +08:00
  • d5c097a2f9 Tiny re-introduce profile id logging (#6912) fzyzcjy 2025-06-07 17:32:50 +08:00
  • 9736cd3b7d [Bugfix] pipeline parallelism and Eagle Qwen2 (#6910) Swipe4057 2025-06-07 11:58:50 +03:00
  • 2f715f51cc Minor compile fused topk (#6944) fzyzcjy 2025-06-07 16:40:38 +08:00
  • d664ca18f2 chore: bump sgl-kernel v0.1.6 (#6943) Yineng Zhang 2025-06-07 00:25:22 -07:00
  • 22fe787852 [sgl-kernel] update deepgemm (#6942) JieXin Liang 2025-06-07 14:24:41 +08:00
  • c4ffbeca19 Add triton fused moe kernel config for E=257 on B200 (#6939) Baizhou Zhang 2025-06-06 23:15:01 -07:00
  • f8eaaab817 [fix] logical_to_all_physical_map index 256 is out of bounds in EP parallel. (#6767) miter 2025-06-07 12:32:33 +08:00
  • 697b0f71f0 [Refactor] image data process in bench_serving (#6879) Xinyuan Tong 2025-06-06 21:11:17 -07:00
  • 132dad874d [PD] Optimize transfer queue forward logic for dummy rank (#6922) shangmingc 2025-06-07 09:26:14 +08:00
  • 60fdad7cf3 Sync the changes on cuda graph runners (#6932) Lianmin Zheng 2025-06-06 18:23:52 -07:00
  • 61ce91ed28 Tiny support customize DeepEP max dispatch tokens per rank (#6934) fzyzcjy 2025-06-07 08:18:35 +08:00
  • e6b7053b60 Fix a bug in abort & Improve docstrings for abort (#6931) Lianmin Zheng 2025-06-06 14:35:45 -07:00