Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 85ec0440a5 Update cutlass dependency. (#5447) Elfie Guo 2025-04-15 23:28:04 -07:00
• 06a1656e02 [doc] Update benchmark_and_profiling.md (#5449) Xiaoyu Zhang 2025-04-16 14:27:34 +08:00
• 6aca583420 Fix several minor issues in PD disaggregation (#5444) Cheng Wan 2025-04-15 23:04:41 -07:00
• 5b5c7237c8 chore: bump v0.4.5.post1 (#5445) Yineng Zhang 2025-04-15 23:00:07 -07:00
• a42736bbb8 Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113) Baizhou Zhang 2025-04-15 22:01:22 -07:00
• dd83e7e9c3 [Bug fix] need record start time in pd mode (#5425) ybyang 2025-04-16 10:11:16 +08:00
• 0769b14bf9 [Minor] Move torch.compile patch to a better place (#5397) Lianmin Zheng 2025-04-15 18:37:07 -07:00
• b64b88e738 [Docs] Update start/install.md (#5398) Michael Yao 2025-04-16 09:12:26 +08:00
• bc24205b32 Support BNB quantization for llama/mllama (#5038) ryang 2025-04-16 09:00:31 +08:00
• 3efc8e2d2a add attention backend supporting matrix in the doc (#5211) mRSun15 2025-04-15 17:16:34 -07:00
• 27a009bb00 Fix ignore_eos parameter when loading a chat template (#5264) Chang Su 2025-04-15 17:09:45 -07:00
• 8ec0bb7d55 chore: upgrade sgl-kernel 0.0.9.post1 (#5436) Yineng Zhang 2025-04-15 15:45:51 -07:00
• fa909dc3c4 feat: update model_specific_adjustment (#5344) Yineng Zhang 2025-04-15 14:45:15 -07:00
• e8f62b20ca BLackwell cutlass mla: Add check for bad page size/block num combinations (#5431) Trevor Morris 2025-04-15 14:07:42 -07:00
• 88defc4d89 fix: solve release issue (#5434) Yineng Zhang 2025-04-15 12:58:11 -07:00
• 6f509d5503 chore: bump sgl-kernel v0.0.9.post1 (#5430) Yineng Zhang 2025-04-15 11:00:21 -07:00
• 12ef7e3bc3 bugfix: fix merge_state_v2 cuda graph (#5419) DefTruth 2025-04-16 01:18:47 +08:00
• 838fa0f218 [minor] cleanup cmakelists.txt (#5420) Lianmin Zheng 2025-04-15 07:07:07 -07:00
• f1b3b75fc6 [PD] Remove unused bootstrap param and fix port table type (#5423) shangmingc 2025-04-15 21:21:20 +08:00
• 33b16ad178 Distinguish bootstrap key only in decode server (#5422) Liangsheng Yin 2025-04-15 20:59:28 +08:00
• ffde65a094 [PD] Fix dynamic port support and MLA buffer for Mooncake (#5415) shangmingc 2025-04-15 19:29:31 +08:00
• 471650dee0 Fix broadcast use cuda device lead to memory capacity unbalanced (#5416) lambert0312 2025-04-15 17:47:26 +08:00
• d06a83fb01 Support dynamic connection and TP 16 (#5351) Yuan Luo 2025-04-15 17:08:07 +08:00
• 5d13440162 [FIX] Fix concatenation error in capture_bs when open --disable-cuda-graph-padding and without MTP (#5412) Zhaoyang Hao 2025-04-15 16:42:27 +08:00
• f88f7e1943 [misc] fix ci flaky case (#5352) JieXin Liang 2025-04-15 16:37:16 +08:00
• 3dfc6023ce Fix bench_serving with random-ids (#5214) Yuhong Guo 2025-04-15 16:34:35 +08:00
• 15e91d721b Tiny fix DeepseekScalingRotaryEmbedding always use forward_native (#5406) fzyzcjy 2025-04-15 16:33:47 +08:00
• 8aab7fdb21 chore: upgrade sgl-kernel 0.0.9 (#5401) Yineng Zhang 2025-04-14 22:37:59 -07:00
• e940dc4f06 chore: bump sgl-kernel 0.0.9 (#5400) Yineng Zhang 2025-04-14 21:34:04 -07:00
• 388e15c0db kernel: support slightly faster merge_state_v2 cuda kernel (#5381) DefTruth 2025-04-15 12:28:23 +08:00
• 11421a3f44 fix: update pr-test-sgl-kernel (#5399) Yineng Zhang 2025-04-14 21:14:59 -07:00
• 6c41fcf0e4 chore: upgrade DeepGEMM (#5395) Yineng Zhang 2025-04-14 20:32:46 -07:00
• ee9d6ca677 [fix/misc] remove duplicate row in deepseek v2 model (#5279) Yangcheng Li 2025-04-15 09:41:24 +08:00
• 2dd6489468 Add H20 dtype fp8_w8a8 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5291) Ximingwang-09 2025-04-15 09:40:31 +08:00
• 61e7c4dd21 Add A800 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5368) lambert0312 2025-04-15 09:39:44 +08:00
• dae7944440 minor clean up of sgl-kernel/CMakeLists.txt (#5393) Lianmin Zheng 2025-04-14 18:38:44 -07:00
• f6772f1497 [Fix] Turn off DeepGEMM by default (#5263) Baizhou Zhang 2025-04-14 17:45:44 -07:00
• ac5b78baf6 fix: update test config (#5392) Yineng Zhang 2025-04-14 17:39:47 -07:00
• 38076dea84 apply fused moe gate in ds v3/r1 (#5371) Xiaoyu Zhang 2025-04-15 07:24:26 +08:00
• 5e0a9b0981 Apply deepseek cuda rope (#5385) Ke Bao 2025-04-15 06:22:43 +08:00
• bdde237562 [perf] experimental enhance fp8 per-tensor quant (#5370) JieXin Liang 2025-04-15 03:35:43 +08:00
• e9fc2ac7b6 [PD Bug] fix MLA get_contiguous_buf_infos error (#5384) ybyang 2025-04-14 22:56:39 +08:00
• 44afde82d7 Fix PD disaggregation bugs (#5326) Liangsheng Yin 2025-04-14 19:27:30 +08:00
• 072df75354 Support for Qwen2.5-VL Model in bitsandbytes Format (#5003) yhyang201 2025-04-14 17:03:40 +08:00
• defede5073 Fix DeepSeek DP Attention + torch compile (#5367) fzyzcjy 2025-04-14 16:07:58 +08:00
• fc72871975 Free metadata_buffer_index after transfer finished (#5364) Yongtong Wu 2025-04-14 16:06:14 +08:00
• 14e8bd889f Free metadata_buffer_index after transfer finished (#5364) Yongtong Wu 2025-04-14 16:04:46 +08:00
• adca585bfb [DeepEP] Reduce routed scaling overhead (#5277) yulei 2025-04-14 07:03:09 +08:00
• 39d90449f3 feat: update experiment_runner (#5360) Yineng Zhang 2025-04-13 15:37:05 -07:00
• 39e411385c fix #5322 (#5359) Yineng Zhang 2025-04-13 13:57:36 -07:00
• 5fbafbb8f8 fix MLATokenToKVPoolHost get_size_per_token bug (#5161) huangtingwei 2025-04-14 03:37:26 +08:00
• a9499885e9 [PD] Add transfer backend abstraction (#5328) Byron Hsu 2025-04-13 10:39:39 -07:00
• f765579046 Fix typo: infight -> inflight (#5357) Liangsheng Yin 2025-04-14 01:25:30 +08:00
• f58b929a51 chore: upgrade sgl-kernel 0.0.8.post3 (#5342) Yineng Zhang 2025-04-13 00:45:59 -07:00
• c1270aabc5 docs: update adoption and sponsorship list with Oracle (#5343) Yineng Zhang 2025-04-12 22:55:25 -07:00
• 8311b07fb9 Fix: Ensure tensors for dist.broadcast match NCCL backend device (#5322) mlmz 2025-04-13 13:50:37 +08:00
• c138025731 misc: update sagemaker Dockerfile (#5341) Yineng Zhang 2025-04-12 22:39:49 -07:00
• b62e7e99b8 feat: adapt merge_state (#5337) Yineng Zhang 2025-04-12 21:14:04 -07:00
• 7d3b7c87f5 fix: determine if flashinfer is installed (#5336) Yineng Zhang 2025-04-12 19:59:13 -07:00
• 75015bb688 ci: update release node (#5333) Yineng Zhang 2025-04-12 14:22:45 -07:00
• b371f7cd36 chore: bump sgl-kernel v0.0.8.post3 (#5332) Yineng Zhang 2025-04-12 12:53:37 -07:00
• 812e82f35e fix: solve cu118 issue for cutlass mla (#5331) Yineng Zhang 2025-04-12 12:51:09 -07:00
• 4879e50c6d [Feat] Add sparse attn to sgl-kernel (#5327) PGFLMG 2025-04-13 02:36:36 +08:00
• bc92107b03 Support server based rollout in Verlengine (#4848) tianlian yi 2025-04-13 01:07:52 +08:00
• 3e4794aad8 refine fused_moe tuning docs (#5294) Xiaoyu Zhang 2025-04-13 01:01:13 +08:00
• 690ec20587 Delete python/sglang/srt/layers/moe/fused_moe_triton/configs/E=257,N=… (#5321) Xiaoyu Zhang 2025-04-13 01:00:03 +08:00
• 2074a2e6b6 Fix: docs/backend/structured_outputs.ipynb (#4884) thyecust 2025-04-12 17:18:55 +08:00
• 57de7c6b5f feat: use fa3 mla by default on hopper (#5210) Yineng Zhang 2025-04-12 01:09:25 -07:00
• 115ae2e728 chore: bump sgl-kernel v0.0.8.post2 (#5317) Yineng Zhang 2025-04-11 23:42:03 -07:00
• aea98512a8 Fix fa3 window size setup (#5316) Qingquan Song 2025-04-11 23:37:52 -07:00
• e4155e96d0 Add flash_attn_varlen_func to sgl-kernel (#5315) Baizhou Zhang 2025-04-11 23:36:36 -07:00
• 1b1b47a949 Fix w8a8_int8 model shared experts fusion load weights error (#5120) lambert0312 2025-04-12 14:33:51 +08:00
• 3c9740d200 update variable naming and comments for rocm (#5299) Zhaoyi Li 2025-04-12 01:15:05 -05:00
• 2eb55770f9 misc: cleanup 3rdparty (#5311) Yineng Zhang 2025-04-11 22:53:50 -07:00
• f65b8d5c89 Blackwell Cutlass MLA kernel (#5142) Trevor Morris 2025-04-11 22:16:51 -07:00
• 5ad0571903 Adjust ci test threshold (#5271) Ke Bao 2025-04-12 13:03:37 +08:00
• 34ef6c8135 [VLM] Adopt fast image processor by default (#5065) Mick 2025-04-12 12:46:58 +08:00
• 611720919d fix: use deepgemm only on hopper (#5310) Yineng Zhang 2025-04-11 20:48:24 -07:00
• 4f288113ce fix: update flash attn (#5308) Yineng Zhang 2025-04-11 16:23:09 -07:00
• 136b8e6afb fix: remove cublas_grouped_gemm (#5307) Yineng Zhang 2025-04-11 16:22:37 -07:00
• 034c5256cc misc: update blackwell Dockerfile (#5306) Yineng Zhang 2025-04-11 15:58:04 -07:00
• c1dd773c19 fix: use fa3 unit test on hopper only (#5304) Yineng Zhang 2025-04-11 15:10:49 -07:00
• 6f8593799b feat: add blackwell workflow (#5303) Yineng Zhang 2025-04-11 13:42:00 -07:00
• f774a0d275 feat: add blackwell Dockerfile (#5302) Yineng Zhang 2025-04-11 13:08:53 -07:00
• 60bcbf2a35 remove moe_align_block_size torch.zeros in small batch/expert mode (#5298) Xiaoyu Zhang 2025-04-12 03:13:55 +08:00
• a0a9f6d64f [Docs] Remove the older supported docs section (#5301) Adarsh Shirawalmath 2025-04-12 00:00:18 +05:30
• 80aa8ca84e fix: update update_wheel_index for cu128 (#5300) Yineng Zhang 2025-04-11 09:31:03 -07:00
• 4aa6bab0b0 [Docs] Supported Model Docs - Major restructuring (#5290) Adarsh Shirawalmath 2025-04-11 21:47:47 +05:30
• c35dcfdb30 [PD] fix: skip warmup request in disaggregation mode to prevent crash on timeout (#5292) Yusong Gao 2025-04-11 23:03:07 +08:00
• c163bf4ff1 chore: bump sgl-kernel v0.0.8.post1 (#5289) Yineng Zhang 2025-04-11 02:11:53 -07:00
• 5598634326 chore: relax the torch version restriction for sgl-kernel compilation (#5288) Yineng Zhang 2025-04-11 02:05:53 -07:00
• b75275b6f2 feat: add cu128 identifier for sgl-kernel (#5287) Yineng Zhang 2025-04-11 01:58:46 -07:00
• 7074e9ca20 fix: enable fp4 compilation on cu128 (#5286) Yineng Zhang 2025-04-11 01:43:44 -07:00
• fc14cca088 Fix a 404 link in send_request.ipynb (#5280) Michael Yao 2025-04-11 16:38:45 +08:00
• e7beff8a13 fix: examples for token_in_token_out_vlm (#5193) XinyuanTong 2025-04-11 01:38:23 -07:00
• 4d2e305149 doc: nested loop code for offline engine (#5244) mlmz 2025-04-11 16:36:30 +08:00
• e53a0b3d5b [fix] fix mrope positions not picked up (#5265) Mick 2025-04-11 16:29:45 +08:00
• 038bc5d521 Support --enable-llama4-multimodal (#5254) Cheng Wan 2025-04-11 04:24:14 -04:00
• aee62d744b Optimize GPU memory usage in FlashAttentionBackend's strided indexing (#5262) Chang Su 2025-04-11 00:34:17 -07:00
• cd7e32e2cb Optimize attention in llama4 (#5127) fzyzcjy 2025-04-11 15:32:41 +08:00