Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 9c138a0445 [3/N] MoE Refactor: Simplify DeepEP Output (#8421) Cheng Wan 2025-07-28 11:37:17 -07:00
• c8f549d96d Fix parsing ChatCompletionMessage (#7273) Timofey 2025-07-28 21:35:14 +03:00
• 134fa43e19 [NVIDIA] Change to use num_local_experts (#8453) Kaixi Hou 2025-07-28 10:38:19 -07:00
• ccfe52a057 fix: update dep (#8467) Yineng Zhang 2025-07-28 10:19:33 -07:00
• 747dd45077 feat: throttle requests at scheduler based on --max_queued_requests (#7565) harrisonlimh 2025-07-28 07:32:33 -07:00
• b582159246 Update PR template (#8465) Ke Bao 2025-07-28 22:12:36 +08:00
• a9dd3ec3e9 fix:reorder topk experts to ensure shared expert replaces minimal score (#8125) erictanjn 2025-07-28 20:36:46 +08:00
• 45bc170b36 chore: bump v0.4.9.post5 (#8458) Yineng Zhang 2025-07-28 02:11:06 -07:00
• 2262369905 Revert "[kernel] opt moe align block kernel by block/warp scan algorithm" (#8457) Xiaoyu Zhang 2025-07-28 16:35:43 +08:00
• fb4ce17de6 Fix per_token_group_quant_8bit when hidden_dim // group_size is not divided by 4. (#8449) strgrb 2025-07-28 16:32:46 +08:00
• 25f73c6cf3 fix GLM4_MOE launch with compressed_tensor quant model (#8456) Minglei Zhu 2025-07-28 01:31:20 -07:00
• 581e7dcb92 GLM-4.5 Model Support Follow-up (#8445) Binyao Jiang 2025-07-27 23:35:20 -07:00
• 484d0e021d doc: add bench_one_batch_server in the benchmark doc (#8441) Qiaolin Yu 2025-07-27 23:07:54 -07:00
• 5922c0cbf6 Remove zstd compression for building Dockerfile.gb200 (#8442) kyleliang-nv 2025-07-27 22:58:53 -07:00
• 6d6a8bc278 GLM-4.5 Model Support (#8224) Yuxuan Zhang 2025-07-28 13:54:07 +08:00
• 2fd5c7049f [PD] Fix abort_request for PD disaggregation (#8352) Shangming Cai 2025-07-28 12:48:27 +08:00
• 4ad9737045 chore: bump transformer to 4.54.0 (#8416) Stefan He 2025-07-27 21:27:25 -07:00
• 2810338401 [feat] Support different attention backends for prefill and decode (#6338) Qiaolin Yu 2025-07-27 20:42:29 -07:00
• fe6a445d1e [router] improve router logs and request id header (#8415) Simo Lin 2025-07-27 19:30:19 -07:00
• dd487e5553 bugfix: Fix XGrammar backend to use model's EOS tokens for constrained generation (#8422) Chang Su 2025-07-27 19:01:02 -07:00
• bb81daefb8 Fix docker buildx push error (#8425) kyleliang-nv 2025-07-27 17:59:38 -07:00
• 58dd95fbc8 Fix test_openai_server (#8419) Chang Su 2025-07-27 13:36:01 -07:00
• b47eda3316 bugfix: Fix multiple finish_reason chunks and tool_calls finish reason check (#8417) Chang Su 2025-07-27 13:31:06 -07:00
• e983d66680 Fix: Improve test_openai_function_calling unit test and fix reasoning_parser.py think_start_token logic (#8316) Binyao Jiang 2025-07-27 13:12:59 -07:00
• b58c3c285e Support ue8m0 for triton quant kernel (#7603) fzyzcjy 2025-07-28 04:04:35 +08:00
• df90645525 Support overlapped lora updates (#8213) Lifu Huang 2025-07-27 13:00:44 -07:00
• 95217a9b4d Change to use native arm runner (#8414) kyleliang-nv 2025-07-27 12:48:12 -07:00
• 22e00eeb4a [Bugfix] Prevent PD server crash from invalid grammar (#8062) Shangming Cai 2025-07-28 00:17:51 +08:00
• b3eac168e7 Support triton kernels v3.4.0 for fused_moe (#8258) Yuan Luo 2025-07-27 17:28:49 +08:00
• 10ee89559e chore: upgrade flashinfer v0.2.9rc2 (#8406) Yineng Zhang 2025-07-27 01:41:22 -07:00
• bf3352c559 chore: update CODEOWNERS (#8407) Yineng Zhang 2025-07-27 01:39:36 -07:00
• 4d921f2b79 [hotfix] fix merge conflicts in FlashInferEPMoE (#8405) Cheng Wan 2025-07-27 01:24:10 -07:00
• 44d600cd67 Support precomputed_embeddings for Llama 4 (#8156) Kevin Xiang Li 2025-07-27 01:14:49 -07:00
• 5c9c275bc8 Use FlashInfer FP4 gemm. (#8241) Elfie Guo 2025-07-27 01:05:22 -07:00
• bf0f448fe5 [2/N] MoE Refactor: Unify weight loader and quant methods (#8397) Cheng Wan 2025-07-27 01:00:21 -07:00
• 36d6f0ba5b fix: fix the missing metrics on non-rank0 nodes (#7720) Yingchun Lai 2025-07-27 15:55:25 +08:00
• 2a1936de96 Add A800 fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#8351) Li Hui 2025-07-27 15:46:07 +08:00
• 2ab97023e3 [router] add different policies for p node and d node (#8395) Simo Lin 2025-07-27 00:39:20 -07:00
• 0bcc195f4e fix: minor fix TransportProxyTensor under tp (#8382) Mick 2025-07-27 15:38:49 +08:00
• 91e3d1542e Update Cutlass in sgl-kernel to v4.1 (#8392) Baizhou Zhang 2025-07-27 00:36:15 -07:00
• 85486b6f6f [NVIDIA] Add Flashinfer MoE blockscale fp8 backend (#8036) Kaixi Hou 2025-07-27 00:34:41 -07:00
• e34cf6ad75 Fix bench script making input data on L2 cache (#7739) fzyzcjy 2025-07-27 15:30:24 +08:00
• 62222bd27e Minor tool for comparison of benchmark results (#7974) fzyzcjy 2025-07-27 15:27:50 +08:00
• ed0fdbf35b Tool to dump and compare internal activation tensors (#7976) fzyzcjy 2025-07-27 15:27:21 +08:00
• b602f42354 Urgent Fix: intern-s1 chat-template matching (#8403) Xinyuan Tong 2025-07-27 00:22:31 -07:00
• 426b74936a Add nvfp4 scaled mm benchmark. (#8401) Qi Yuhang 2025-07-27 14:18:04 +08:00
• 528bd1ed85 HiCache, check before terminate prefetching (#8372) Zhiqiang Xie 2025-07-26 23:13:16 -07:00
• 62a6b7c773 Add docker release flow for gb200 (#8394) kyleliang-nv 2025-07-26 21:25:07 -07:00
• 761546315c Remove slot usage in code to be backward-compatible with python 3.9 (#8396) Lifu Huang 2025-07-26 21:24:22 -07:00
• 5c705b1dce Add perf tests for LoRA (#8314) Lifu Huang 2025-07-26 14:55:22 -07:00
• b7094a5ef1 model: support intern-s1 (#8350) RunningLeon 2025-07-27 04:48:51 +08:00
• da0c026084 Tiny assert EPLB is used together with expert parallel (#8381) fzyzcjy 2025-07-26 18:20:39 +08:00
• 3212c2ad3f vlm: optimize tensor transport (#6003) Mick 2025-07-26 17:41:01 +08:00
• 534756749a chore: improvements on mm_utils (#7737) Mick 2025-07-26 17:38:56 +08:00
• ce32bc2ba9 Extract update_weights from RL Engine to SGLang to keep simplicity and fix torch reduce (#8267) Stefan He 2025-07-26 02:00:59 -07:00
• e236d8fee8 Save peak memory in logits processor (#8343) Cheng Wan 2025-07-26 01:46:42 -07:00
• 4fa44d63c6 chore: improve mmmu benchmark (#7000) Mick 2025-07-26 16:19:45 +08:00
• e6312d271d Uodate Dockerfile.gb200 to latest sglang (#8356) kyleliang-nv 2025-07-26 00:22:06 -07:00
• 8af145b7dc Fix test_moe_fused_gate_combined sgl-kernel ci test (#8374) Ke Bao 2025-07-26 09:30:12 +08:00
• 2272c2a5b5 chore: bump v0.4.9.post4 (#8305) Yineng Zhang 2025-07-25 17:12:47 -07:00
• 3ec0b21229 [CI] Fix flaky threshold (#8370) Lianmin Zheng 2025-07-25 16:41:56 -07:00
• 58c468f404 Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333) Trevor Morris 2025-07-25 16:40:23 -07:00
• f8ca2368b2 fix: kimi k2 xgrammar crash (#8367) Yineng Zhang 2025-07-25 15:44:01 -07:00
• d8ee15643b [Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 (#8363) Chang Su 2025-07-25 14:59:42 -07:00
• 7181ec8cfc fix: upgrade nccl version (#8359) Yineng Zhang 2025-07-25 14:59:02 -07:00
• ed2e313eb6 Clean up server_args, triton cache manager (#8332) Lianmin Zheng 2025-07-25 14:14:51 -07:00
• f8260f2539 [Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357) Chang Su 2025-07-25 12:03:16 -07:00
• 12cb760a37 Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#8344) Xu Wenqing 2025-07-26 01:58:12 +08:00
• 1b9cea5ade [P/D] Support ipv6 in P/D scenario (#7858) Stepan Kargaltsev 2025-07-25 18:53:30 +03:00
• 9045cc1eb8 [torch.compile bug] avoid biased_grouped_topk_impl func repeatedly triggering torch.compile in forward pass (#8353) Xiaoyu Zhang 2025-07-25 21:17:47 +08:00
• 70e37b97bf chore: upgrade mooncake 0.3.5 (#8341) Shangming Cai 2025-07-25 16:17:26 +08:00
• 15d2759174 [CPU] Add tutorial docs for SGL on CPU (#8000) Zaili Wang 2025-07-25 15:03:16 +08:00
• af4b9bae95 [AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs (#7135) Hubert Lu 2025-07-24 23:44:28 -07:00
• 7ad6b766c5 fix: Fix failed functional tests https://github.com/meta-llama/llama-stack-evals (#8266) Ying Wang 2025-07-24 23:11:32 -07:00
• c0fb25e949 DP Enhancement (#8280) Cheng Wan 2025-07-24 21:36:21 -07:00
• 28d4d47280 [Feature] Integrate quick allreduce and select the best allreduce implementation (#6619) li haoyang 2025-07-25 11:48:42 +08:00
• f4674df646 support idle batch for TBO (#8233) ZhichenJiang 2025-07-25 11:43:52 +08:00
• d40846d456 breakdown kernel update (#8334) Zhiqiang Xie 2025-07-24 17:33:17 -07:00
• 145482f422 HiCache Storage TP Refinement (#8307) Zhiqiang Xie 2025-07-24 17:31:47 -07:00
• 39fe1e880d [router] add request format unit test (#8300) Simo Lin 2025-07-24 14:30:37 -07:00
• 33c4b4d04e [router] add streaming unit test (#8299) Simo Lin 2025-07-24 14:30:27 -07:00
• 8d1c5b948e chore: upgrade flashinfer v0.2.9rc1 (#8301) Swipe4057 2025-07-25 01:29:56 +04:00
• a167fd0bcb [code style] Clean dead triton kernel code in fused_moe and useless vllm_ops import (#8310) Xiaoyu Zhang 2025-07-24 14:38:30 +08:00
• 2f86f3ad62 [router] add endpoint unit test (#8298) Simo Lin 2025-07-23 23:26:44 -07:00
• bfb118c01e fix bug when eos_ids==0 (#8315) Minho Ryu 2025-07-24 15:18:47 +09:00
• f6e07f2796 [router] fix pd model completion request (#8303) Simo Lin 2025-07-23 23:18:29 -07:00
• 5dd0f870ab [bug] fix pd completion protocol for batching support (#8317) Simo Lin 2025-07-23 23:18:17 -07:00
• f7e102d56a Pin the version of petit kernel to fix the APIs (#8235) Haohui Mai 2025-07-23 17:57:20 -07:00
• 0e5fa67773 [AMD] Pull latest image for AMD CI (#8070) michael-amd 2025-07-23 17:56:14 -07:00
• 624a3b8d1f Fix incomplete tool call capture issue in streaming response of DeepSeek-V3 when enable MTP (#7562) xianzhiT 2025-07-24 08:40:23 +08:00
• 01079e174f feat(function call): complete utility method for KimiK2Detector and enhance documentation (#8043) Chang Su 2025-07-23 17:37:31 -07:00
• 0e7a5b2694 fix: prevent crashes due to logit bias dimension mismatch (#7685) J 2025-07-23 15:30:55 -07:00
• 4953f4ca9a chore: upgrade sgl-kernel 0.2.7 (#8304) Yineng Zhang 2025-07-23 15:07:27 -07:00
• 38000a5f44 Fix gemma3n with hybrid swa (#8240) Xinyuan Tong 2025-07-23 13:29:18 -07:00
• 70251e935e fix: match chat-template for internvl3 (#8262) Xinyuan Tong 2025-07-23 13:29:03 -07:00
• c87d4fec99 Fix the issue of incorrect finish reason in final stream response chunk returned during tool call (#7708) xianzhiT 2025-07-24 04:28:53 +08:00
• a99801e075 [Performance][PD Disaggregation] optimize TokenToKVPoolAllocator by sorting free pages (#8133) YiXR 2025-07-24 04:28:12 +08:00
• 4c605235aa fix: workaround for deepgemm warmup issue (#8302) Yineng Zhang 2025-07-23 12:01:51 -07:00
• 6f8f4aeea4 [router] add common ut infra to mock worker and app (#8295) Simo Lin 2025-07-23 10:07:51 -07:00
• 0c8dab9e67 [sgl-kernel] Opt per_token_quant_fp8 with warp reduce (#8130) Yuan Luo 2025-07-23 21:22:59 +08:00