Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 11b23ae97b Remove extra copy in deepseek forward absorb (#5578) Ke Bao 2025-04-22 10:33:21 +08:00
• b9c87e781d chore: bump v0.4.5.post3 (#5611) Yineng Zhang 2025-04-21 18:16:20 -07:00
• 968ef51562 Support aiter RMSNorm in AMD (#5510) michael-amd 2025-04-21 17:40:39 -07:00
• 1343200299 Clean up mem settings (#5610) Lianmin Zheng 2025-04-21 17:19:00 -07:00
• c2942907d5 [feature] enable pre compile jit deep_gemm (#5580) JieXin Liang 2025-04-22 07:52:53 +08:00
• e69a219074 Enhance GPU memory settings (#5604) Liangsheng Yin 2025-04-22 06:15:00 +08:00
• bf98d2e377 [PD] Support prefill overlap + Ensure no race condition (#5609) Byron Hsu 2025-04-21 12:12:56 -07:00
• e65b9f21e3 [PD] Support decode overlap schedule (#5608) Byron Hsu 2025-04-21 12:06:16 -07:00
• 4dce1cc608 [PD] Add NIXL transfer backend (#5477) Trevor Morris 2025-04-21 10:36:12 -07:00
• deded17f38 [PD] Fix edge case and simplify large page size + chunked prefill (#5589) Byron Hsu 2025-04-21 10:27:02 -07:00
• f29a718f63 [PD] Fix generate endpoint of min_lb for PD (#5598) shangmingc 2025-04-21 21:39:18 +08:00
• 3f57b00a59 Support PD bootstrap fields on /v1/chat/completions endpoint (#5488) Yongtong Wu 2025-04-21 16:10:58 +08:00
• 453d412cdb Tiny update error hint (#5037) fzyzcjy 2025-04-21 15:47:47 +08:00
• dc86f25a57 Tiny remove duplicated code (#5021) fzyzcjy 2025-04-21 15:47:32 +08:00
• 08289eaa3e Support o1 model on Azure (#4980) Chuyue Sun 2025-04-21 00:46:09 -07:00
• 3b6d539f63 [Fix] Enhance DP Attention for IPv6 Compatibility (#4937) Lucius 2025-04-21 15:44:11 +08:00
• 57131dd955 [Feat.] Enable grafana to show metrics (#4718) Huapeng Zhou 2025-04-21 15:43:42 +08:00
• a7591ecf69 Update protocol.py to fix #4589 (#4590) moontidef 2025-04-21 15:42:50 +08:00
• c44f2869c9 Modify metrics service endpoint (#3443) lambert0312 2025-04-21 15:35:38 +08:00
• 685d8980c3 Tiny add warning when cannot recognize bool env var (#5348) fzyzcjy 2025-04-21 14:11:29 +08:00
• 70645f4d7d upstream hicache fixes (#5570) Zhiqiang Xie 2025-04-20 23:08:30 -07:00
• 188f0955fa Add Speculative Decoding Eagle3 topk > 1 (#5318) Qingquan Song 2025-04-20 22:58:28 -07:00
• eef9433b46 Fix flush cache (#5590) Lianmin Zheng 2025-04-20 22:56:40 -07:00
• 97cb762bb6 [misc] remove is_cuda_available (#5319) JieXin Liang 2025-04-21 09:16:51 +08:00
• 1195182040 Tiny add Engine.flush_cache API (#5241) fzyzcjy 2025-04-21 09:15:03 +08:00
• 5239d79568 Speedup shared expert weight construction by avoid cloning (#5188) fzyzcjy 2025-04-21 09:12:01 +08:00
• f08154193c Perform Batch Tokenization. (#5141) Sundara Raman Ramachandran 2025-04-20 18:10:37 -07:00
• 2b3bdc938e Correct grafana heatmap. (#5019) mac0ne 2025-04-21 08:58:56 +08:00
• 5fc4b6004e Add sanity check for max_running_requests (#5016) fzyzcjy 2025-04-21 08:56:49 +08:00
• b868526d94 Fix one more issue reported by torchfix (#4859) Brayden Zhong 2025-04-20 20:49:27 -04:00
• 502524e2da compressed_tensors: port w8a16 fp8 from vllm (#4852) Juwan Yoo 2025-04-21 09:48:31 +09:00
• 4c7640079c check marlin format before attempting conversion (#4675) Enrique Shockwave 2025-04-21 01:47:09 +01:00
• 9f3bd2ad39 Feat: Implement JSON Mode (response_format.type="json_object") (#4733) kyle-pena-kuzco 2025-04-20 20:41:22 -04:00
• 8de53da989 smaller and non gated models for docs (#5378) simveit 2025-04-21 02:38:25 +02:00
• fac17acf08 add function call parser for DeepSeek V3 (#5224) Yi Zhou 2025-04-21 08:38:08 +08:00
• 8b39274e34 [Feature] Prefill assistant response - add continue_final_message parameter (#4226) Adarsh Shirawalmath 2025-04-21 06:07:18 +05:30
• 5156d5a413 Add test config yamls for Deepseek v3 (#5433) Baizhou Zhang 2025-04-20 17:28:52 -07:00
• c951d312ed [PD] Fix large page size + chunk prefill (#5588) Byron Hsu 2025-04-20 17:21:54 -07:00
• dcb8232596 Fix ChatCompletionMessageGenericParam to allow for None content (#5452) AmadeusW 2025-04-21 08:15:38 +08:00
• 66c0ff9e31 fix: use fa3 for gemma2 (#5586) Yineng Zhang 2025-04-20 17:02:09 -07:00
• 9a7e83e899 Fix enable chunked prefill for Llama4 (#5575) tarinkk 2025-04-20 20:01:30 -04:00
• 417b44eba8 [Feat] upgrade pytorch2.6 (#5417) lukec 2025-04-21 07:06:34 +08:00
• 475e2e378a [PD] Fix server crash when using batch requests (#5531) fzyzcjy 2025-04-21 07:02:23 +08:00
• fba86b6b54 Tiny improve error message (#5526) fzyzcjy 2025-04-21 07:00:15 +08:00
• 072b4d0398 Add document for LoRA serving (#5521) Baizhou Zhang 2025-04-20 14:37:57 -07:00
• 9c43477710 Super tiny fix typo (#5559) fzyzcjy 2025-04-21 05:21:18 +08:00
• fa2f677e18 Fix torch memory saver not enabled in DP scenario (#5560) fzyzcjy 2025-04-21 05:20:52 +08:00
• 463d4b7400 Fix DeepEP cannot run on latest master (#5567) fzyzcjy 2025-04-21 05:19:42 +08:00
• 9924bbe153 Fix bench_serving fail when zero warmup requests (#5574) fzyzcjy 2025-04-21 05:16:03 +08:00
• fbdc94ba59 Release v0.4.5.post2 (#5582) Lianmin Zheng 2025-04-20 14:12:37 -07:00
• b54b5a96e4 [Doc]Add instruction for profiling with bench_one_batch (#5581) Baizhou Zhang 2025-04-20 14:05:36 -07:00
• bca832c7c6 [Fix] fix outlines and xgrammar (#4947) JieXin Liang 2025-04-21 04:31:25 +08:00
• d9dd529854 enable DeepSeek V3 shared_experts_fusion in sm90 (#5571) Xiaoyu Zhang 2025-04-21 03:46:42 +08:00
• 0a0dd34e6a Fix BumpAllocator error when no input_ids (#5564) fzyzcjy 2025-04-20 17:20:53 +08:00
• 80ac527d22 [PD] Fix DeepSeek cannot be run on latest master (#5568) fzyzcjy 2025-04-20 17:19:48 +08:00
• 99456bcacb [perf] introduce deep gemm group_gemm_masked as bmm (#5432) JieXin Liang 2025-04-20 15:38:27 +08:00
• d07e797ace Fix bench_one_batch producing unnatural results for expert parallel (#5149) fzyzcjy 2025-04-20 15:38:04 +08:00
• c555d794f7 Minor update for ROCm variable style (#5562) Zhaoyi Li 2025-04-20 01:45:27 -05:00
• e2574ee986 fix hicache write back (#5543) Zhiqiang Xie 2025-04-19 21:56:22 -07:00
• ab4b5606e4 [PD] Support page size > 1 (#5561) Byron Hsu 2025-04-19 21:54:27 -07:00
• 20f1c8e374 Fix sampler nan check when calling top_k_top_p_sampling_from_probs (#5546) Yubo Wang 2025-04-19 21:47:23 -07:00
• 613b197e57 Remove one kernel in per_tensor_quant_mla_fp8 (#5549) fzyzcjy 2025-04-20 06:08:15 +08:00
• d58e354472 simplify the control logic for using shared experts fusion (#5504) Xiaoyu Zhang 2025-04-20 04:17:35 +08:00
• bf86c5e990 restruct compressed_tensors_w8a8_fp8 (#5475) Xiaoyu Zhang 2025-04-19 19:52:15 +08:00
• dca90f1db8 [PD] Remove the requirement of config file for mooncake backend (#5460) shangmingc 2025-04-19 19:31:00 +08:00
• 0961feefca feat: use flashinfer jit package (#5547) Yineng Zhang 2025-04-19 00:28:39 -07:00
• 59dd090f1c [PD] Fix no cache connect for recevier (#5534) ybyang 2025-04-19 14:55:28 +08:00
• 569b032c58 [PD] Tiny fix timeout error when generate (#5545) fzyzcjy 2025-04-19 14:42:57 +08:00
• f6a71139a8 Make profiler output file names consistent (#5548) fzyzcjy 2025-04-19 13:57:11 +08:00
• 1e0806f30b Fix DeepGEMM masked cannot be run on groups not being multiple or 4 (#5340) fzyzcjy 2025-04-19 13:38:07 +08:00
• 2c11f9c2eb chore: upgrade sgl-kernel 0.0.9.post2 (#5540) Yineng Zhang 2025-04-18 21:17:23 -07:00
• a6f892e5d0 Revert "Avoid computing lse in Ragged Prefill when there's no prefix.… (#5544) Yineng Zhang 2025-04-18 16:50:21 -07:00
• 08b518d51f fix util import (#5542) Yineng Zhang 2025-04-18 15:06:46 -07:00
• 4db463b1ad [Model] Adding Qwen3 and Qwen3MoE (#4693) yhyang201 2025-04-19 00:51:29 +08:00
• bfa3922451 Avoid computing lse in Ragged Prefill when there's no prefix. (#5476) Wenxuan Tan 2025-04-18 03:13:57 -05:00
• e465b08ddb fix bug of VLLM_AVAILABLE not defined (#5497) liwenju0 2025-04-18 15:59:03 +08:00
• bed05878f6 fix kimi vl running bug after rebase main (#5461) Xiaoyu Zhang 2025-04-18 15:17:34 +08:00
• b2a189dd11 use sglang_per_token_group_quant_fp8 from sgl-kernel instead of trion kernel (#5473) strgrb 2025-04-18 15:05:24 +08:00
• f28d82997a chore: bump sgl-kernel 0.0.9.post2 (#5518) Yineng Zhang 2025-04-17 23:42:39 -07:00
• 8e09b37077 Sgl kernel fused_moe_gate support n_shared_experts (#5440) Xiaoyu Zhang 2025-04-18 14:05:15 +08:00
• 53dcf38876 Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836) fzyzcjy 2025-04-18 12:38:26 +08:00
• 1effba4c70 Configuration qwen2_moe.py - qkv_bias now in transformers (#5512) Michael Feil 2025-04-17 21:23:22 -07:00
• a0fc5bc144 [docs] Fix several consistency issues in sampling_params.md (#5373) Michael Yao 2025-04-18 10:54:40 +08:00
• 27e9538a7e Fix: fix the exception 'the memory capacity is unbalanced. Some GPUs … (#5426) mlmz 2025-04-18 10:51:39 +08:00
• 211c7b31b8 Fix: Incorrect parameters passed to forward_batch_generation (#5506) (#5511) u4lr451 2025-04-18 09:49:59 +08:00
• c08a717c77 [Feat] Update sgl-kernel flashinfer to latest main version (#5500) PGFLMG 2025-04-18 03:43:23 +08:00
• f13d65a7ea Doc: fix problems of the 'Execute Notebooks / run-all-notebooks' ci caused by the unstability of deepseek-ai/DeepSeek-R1-Distill-Qwen-7B (#5503) mlmz 2025-04-18 02:37:43 +08:00
• 06d0a3d92b [Bug fix] use correct func path in deepseek (#5496) Xuchun Shang 2025-04-17 17:41:41 +08:00
• 22c2a79dc5 Fix a link in sgl-kernel/README.md (#5493) Michael Yao 2025-04-17 17:25:28 +08:00
• 8beb356f0d Refactor DeepSeek decoder layer branches (#5205) fzyzcjy 2025-04-17 17:11:11 +08:00
• c776234b45 Enable local attention during decode (#5479) Chang Su 2025-04-17 02:07:43 -07:00
• 3bface15e6 Feat/support encoder model (like bert) (#4887) woodx 2025-04-17 16:50:48 +08:00
• 6fb29ffd9e Deprecate enable-flashinfer-mla and enable-flashmla (#5480) Baizhou Zhang 2025-04-17 01:43:33 -07:00
• 4fb05583ef Deprecate disable-mla (#5481) Baizhou Zhang 2025-04-17 01:43:14 -07:00
• 81c891111f Add test for flash_attn_varlen_func kernel (#5484) Baizhou Zhang 2025-04-17 01:42:56 -07:00
• 92d1561b70 Update attention_backend.md: plural form (#5489) Didier Durand 2025-04-17 10:42:40 +02:00
• 8f783c1943 [Model Support] unsloth/Phi-4-mini bnb model (#4982) eigen 2025-04-16 22:58:20 -04:00
• 90faf9018e [verl] Modify the update_weights func to align with verl's resharding (#5345) BearBiscuit 2025-04-17 10:56:57 +08:00
• 177320a582 Clean up imports (#5467) Lianmin Zheng 2025-04-16 15:26:49 -07:00
• d7bc19a46a add multi-lora feature in README.md (#5463) Ying Sheng 2025-04-16 03:25:25 -07:00