Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 38216cf049 concurrently load weights of DeepseekV2ForCausalLM (#7943) Albert 2025-07-16 04:41:19 +08:00
• 4a8837950a fix: resolve arm build issue (#8052) Yineng Zhang 2025-07-15 01:19:34 -07:00
• f1f1d1d40d Fix the input tools format and history tool_calls in OpenAI API (#6556) jiawei 2025-07-15 15:58:55 +08:00
• 9120e83d03 fix: remove redundant rotary embedding cache recomputation in MiniCPM (#8022) Xinyuan Tong 2025-07-15 00:12:45 -07:00
• 6e923dbd30 feat: update multimodal data handling in engine entrypoint (#8002) Xinyuan Tong 2025-07-15 00:12:22 -07:00
• c268c11c71 [feat]Support fusion kernel for constructing quant input and scale factor for fp8_blockwise_scaled_grouped_mm (#8023) Qi Yuhang 2025-07-15 15:02:44 +08:00
• e6d5988442 Update CODEOWNERS (#8044) Chang Su 2025-07-14 23:38:15 -07:00
• 9b560c3e1c fix: modality length mismatch with image_data (#7887) 杨睿 2025-07-15 14:27:54 +08:00
• 5dc5866e8e Setup workflow for releasing mi300x and mi350x dockers. (#8035) Sai Enduri 2025-07-14 21:51:43 -07:00
• 64e78bb31b prevent server crash from potential invalid grammar (#7897) ehuaa 2025-07-15 11:21:45 +08:00
• 7c39e8a198 Fix Bug 'get_cpu_copy not Implemented' in pd offloading mode (#7982) hzh0425 2025-07-15 05:57:10 +08:00
• d969504d9a Fix flaky CI: test_vlm_models (#8006) Lifu Huang 2025-07-14 14:56:41 -07:00
• 1ebec1a8b0 [Feature] CUDA Green Context Support (#7649) ykcombat 2025-07-15 02:49:16 +08:00
• d4d0c7c367 [Feature]TP Group Switching for PD-Multiplexing (#7653) ykcombat 2025-07-15 02:35:46 +08:00
• 8d2cf38c79 [Minor] Remove redundant print (#8005) Lianmin Zheng 2025-07-14 10:55:13 -07:00
• 2117f82def [ci] CI supports use cached models (#7874) Hank Han 2025-07-14 19:42:21 +08:00
• c07f647c9f perf: add kimi k2 fused_moe tuning config for h30_3e (#8021) Yusong Gao 2025-07-14 17:56:11 +08:00
• 07452cbe8e [CPU] fix no attribute 'can_fuse_mlp_allreduce' error (#8010) Chunyuan WU 2025-07-14 16:32:43 +08:00
• a562c8a35c [Dockerfile] Multi-arch support for ROCm (#7902) mqhc2020 2025-07-14 14:13:09 +08:00
• cb736df854 Support for Phi-1.5 & Phi-2 models (#7862) Praneth Paruchuri 2025-07-14 07:13:40 +05:30
• e2ed9d049a Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes (#7844) Lifu Huang 2025-07-13 18:36:01 -07:00
• b5dd5e8741 chore: remove unnecessary limits on quantization methods in test script (#7997) Peng Zhang 2025-07-14 07:11:49 +08:00
• 9379da77de SWA Prefix Cache (#7367) Hanming Lu 2025-07-13 12:31:07 -07:00
• 0c55cbcfc5 [BugFix] add verify logit_bias to avoid crash because of IndexError (#7749) ehuaa 2025-07-14 02:44:12 +08:00
• c46e069d34 Tiny fix mooncake log warning wrong output (#7952) fzyzcjy 2025-07-13 12:22:44 +08:00
• 42fc44100a [minor] Add server_args check for Llama4 with hybrid (#7988) Ying Sheng 2025-07-12 20:13:40 -07:00
• 5f6756b038 [BugFix] fix pre_reorder_triton_kernel default int32 issue (#7814) Morpheus Guo 2025-07-13 04:42:36 +08:00
• 98aa836bbf Overlap the gating function with shared experts in DeepSeek (#7978) Cheng Wan 2025-07-12 13:41:50 -07:00
• 22bd857cb5 docs: update README (#7985) Yineng Zhang 2025-07-12 13:31:11 -07:00
• ccfa084125 [script] update loogle test (#7975) Ying Sheng 2025-07-12 00:06:17 -07:00
• bcc5ba94b4 [minor fix] SWA missing methods (#7972) Ying Sheng 2025-07-11 23:57:02 -07:00
• cee9f329c4 [minor fix] llama4 hybrid memory (#7950) Ying Sheng 2025-07-11 23:11:36 -07:00
• eb118d88c4 chore: bump v0.4.9.post2 (#7963) Yineng Zhang 2025-07-11 21:11:20 -07:00
• 732fc8e405 chore: upgrade sgl-kernel 0.2.5 (#7971) Yineng Zhang 2025-07-11 20:35:06 -07:00
• f2d5c4920e [router] add worker abstraction (#7960) Simo Lin 2025-07-11 20:17:48 -07:00
• 2a2d3478af Fix wrong gemm branch cause 250us slower (#7969) fzyzcjy 2025-07-12 10:45:09 +08:00
• aa2056091a delete uselese code caused by fuse allreduce+add_rmsnorm pr (#7970) Xiaoyu Zhang 2025-07-12 10:43:38 +08:00
• 61bb285827 chore: upgrade xgrammar 0.1.21 (#7962) Yineng Zhang 2025-07-11 19:26:52 -07:00
• 880221bd3b Revert "[PD Disaggregation] replace transfer with batch transfer for better performance (#7236)" (#7968) fzyzcjy 2025-07-12 10:03:01 +08:00
• 8f3173d0b0 chore: bump sgl-kernel v0.2.5 (#7964) Yineng Zhang 2025-07-11 18:24:20 -07:00
• 26118a133d [fix]Update unitest for fp8_blockwise_scaled_grouped_mm kernel (#7932) Qi Yuhang 2025-07-12 05:29:13 +08:00
• 475a249bb8 temporarily disable deepep-8-gpu and activate two small tests (#7961) Cheng Wan 2025-07-11 14:22:05 -07:00
• 191d836ff6 fix: minor fix for modelopt weight load compatibility (#7953) Peng Zhang 2025-07-12 05:20:58 +08:00
• 86044712c6 [feature] kv transfer support of ascend npu (#7795) ronnie_zheng 2025-07-11 10:07:51 +03:00
• 615553079d Support Kimi K2 (#7940) Atream 2025-07-11 15:02:21 +08:00
• 49a5915f53 [ready b200] fuse allreduce+add_rmsnorm in prepare_attention + mlp module (#7775) Xiaoyu Zhang 2025-07-11 06:12:39 +08:00
• 766392c6bd [feature]Ascend quantization support (#7791) ronnie_zheng 2025-07-10 19:17:37 +03:00
• 4a0d19198b Fix bug of deepseek-v3 under DP+EP mode with large batchsize/seqlen (#6449) likesen-alibaba 2025-07-10 16:19:56 +08:00
• 5748241549 add sentencepiece as dependency explicitly (#7922) Zaili Wang 2025-07-10 16:06:27 +08:00
• 2d54d4bb64 Feat: Support Phi-3.5-MoE in SGLang (#7907) Binyao Jiang 2025-07-09 23:51:33 -07:00
• b5e3d6031c vlm: support video as an input modality (#5888) Mick 2025-07-10 14:48:35 +08:00
• 4ed57807c2 [bugfix] add pd router policy validation (#7904) Simo Lin 2025-07-09 21:52:06 -07:00
• dd445a41f5 [feature] Add start step profile argument in /start_profile (#7608) kyleliang-nv 2025-07-09 18:42:15 -07:00
• 7590f5224b [router] Update metrics when request completes (#7899) Shuaiyi Zhang 2025-07-09 23:18:09 +08:00
• f9df11ae86 Remove unused imports (#7898) almaslof 2025-07-09 17:36:48 +03:00
• d389bedf72 [CPU][Qwen3 MoE] Enable fused_topk CPU fusion and enhance FP8 TP padding (#7838) jianan-gu 2025-07-09 17:04:21 +08:00
• ac80f4da57 [CPU] [FP8] set SGLANG_CPU_FP8_CVT_FTZ in CMakeLists.txt (#7885) Chunyuan WU 2025-07-09 16:53:53 +08:00
• d487555f84 [CI] Add deepep tests to CI (#7872) Cheng Wan 2025-07-09 01:49:47 -07:00
• e5888eddda Fixes typo in assertion message (#7895) Xinyuan Tong 2025-07-09 01:47:14 -07:00
• 066f4ec91f chore: bump v0.4.9.post1 (#7882) Yineng Zhang 2025-07-09 00:28:17 -07:00
• b6b6268ccf Revert "Embedding parallel by attn_tp (#7623)" (#7880) Yineng Zhang 2025-07-08 22:03:09 -07:00
• 0870232195 Update native_api doc to match the change in the get_model_info endpoint (#7660) Yikai Zhang 2025-07-09 12:05:58 +08:00
• 64c5907e12 [PD] Add guidance for prefill bootstrap timeout (#7846) Shangming Cai 2025-07-09 12:00:34 +08:00
• 128f16a817 [CPU]convert topk_weights to fp32 for INT8 and FP8 paths (for llama4) and fix LmHead weight pack (#7818) Chunyuan WU 2025-07-09 10:27:24 +08:00
• 4986104618 Bump xgrammar's version to 0.1.20 (#7866) ybyang 2025-07-09 08:55:30 +08:00
• a37e1247c1 [Multimodal][Perf] Use pybase64 instead of base64 (#7724) Brayden Zhong 2025-07-08 17:00:58 -04:00
• 136c6e0431 fix: Handles input_embeds in GenerateReqInput when n>1 (#7830) Xinyuan Tong 2025-07-08 14:00:42 -07:00
• 43e20c0647 Support Mimo-VL (#7579) Xinyuan Tong 2025-07-08 14:00:25 -07:00
• 4bab50a6b5 Fix llama4 vision (#7840) Xinyuan Tong 2025-07-08 14:00:03 -07:00
• 2e7ab862e3 Fix illegal memory in trtllm allreduce fusion (#7864) Xiaoyu Zhang 2025-07-09 02:47:17 +08:00
• 51ae40306a [router] forward stream_options in request (#7860) Shuaiyi Zhang 2025-07-08 23:03:38 +08:00
• 653b873b91 Fix cache modules of triton import error (#7832) kk 2025-07-08 17:50:09 +08:00
• d379bda4fa [Bugfix] Fix two batch overlap with auto DeepEP Dispatch (#7853) Shangming Cai 2025-07-08 17:49:32 +08:00
• 2b0e1d1ce0 [Minor] Fix sporadic CI timeout caused by underestimated tests. (#7850) Lifu Huang 2025-07-08 01:01:49 -07:00
• 659907e32b Enable ModelOpt Llama4 fp8 checkpoint deployment in SGLang (#7129) Zhiyu 2025-07-08 00:19:50 -07:00
• cb9d91ea8a feat: support DeepSeek-R1-W4AFP8 model with ep-moe mode (#7762) SijiaYang 2025-07-08 05:47:21 +08:00
• 6a6e0bb7fd docs: update README (#7821) Yineng Zhang 2025-07-07 02:47:04 -07:00
• 076313bd09 [AMD] Fail gracefully when AITER is unavailable gfx90a GPUs (#7187) Haohui Mai 2025-07-07 02:09:58 -07:00
• 9abe1163ac fix duplicate args in schedule_batch (#7816) Ziming Huang 2025-07-07 16:31:03 +08:00
• 3646f6bb3e [misc] release new router version (#7798) Simo Lin 2025-07-06 22:54:17 -07:00
• 35724aa182 [docs] update router readme (#7797) Simo Lin 2025-07-06 22:54:11 -07:00
• 2fc824b84c Kernels for efficient KV cache IO (#7313) Zhiqiang Xie 2025-07-06 22:53:36 -07:00
• 253454de9b Integrate triton moe kernel (#7689) Yuan Luo 2025-07-07 11:05:49 +08:00
• ea3e7ffec7 [bugfix] Fix sgl-router get_server_info endpoint compatibility issue (#7813) Simo Lin 2025-07-06 19:52:57 -07:00
• 8d4a01cbd7 Log the timestamps of each prefill/decode iteration (#6094) yuhsuan-t 2025-07-06 18:57:27 -07:00
• a3398d8478 Optimize moe align block size kernel (#7794) Ke Bao 2025-07-07 09:20:30 +08:00
• ba69c153f6 [RL]: Fix error tagging in multi-stage wake up (#7812) Nan Jiang 2025-07-06 16:51:29 -07:00
• 3589aa79b0 [RL] Fix illegal memory for _import_static_state (#7733) Stefan He 2025-07-06 16:25:21 -07:00
• e00715eb66 [AMD] Add test_fused_moe.py and test_rope_rocm.py to AMD CI (#5246) Hubert Lu 2025-07-06 01:47:16 -07:00
• ea4bf12286 Fix division-by-zero bug in LoRA triton kernels. (#7785) Lifu Huang 2025-07-06 00:45:29 -07:00
• a291439a59 Support logprobs in two-batch overlap (#7709) fzyzcjy 2025-07-06 10:05:32 +08:00
• 54411f6afa fix: disable dsv3_router_gemm in dsv3_nextn (#7793) JieXin Liang 2025-07-06 10:01:01 +08:00
• 625018d259 fix: free disk space (#7803) Yineng Zhang 2025-07-05 18:52:25 -07:00
• 5732d904cc [misc] remove pdlb rust (#7796) Simo Lin 2025-07-05 17:44:51 -07:00
• ec5f9c6269 chore: bump v0.4.9 (#7802) Yineng Zhang 2025-07-05 17:40:29 -07:00
• 62f5522ffe chore: upgrade sgl-kernel v0.2.4 (#7801) Yineng Zhang 2025-07-05 17:37:40 -07:00
• 01f9873048 Fix CI test OOM issue. (#7799) Lifu Huang 2025-07-05 15:11:02 -07:00
• 199d621845 ci: fix port args (#7792) Mick 2025-07-06 06:06:42 +08:00
• f200af0d8c chore: bump sgl-kernel v0.2.4 (#7800) Yineng Zhang 2025-07-05 15:03:31 -07:00
• 5589b75024 Add treemask mode to build_eagle_tree & release sgl-kernel 0.2.3 (#7756) Lianmin Zheng 2025-07-05 12:17:05 -07:00