Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 9c53dad809 Fix MTP MoE weight loading with NVFP4 target model. (#10758) Jue WANG 2025-09-22 17:21:57 -04:00
• 7ca1bea63d [router] update ci so only execute benchmarks when labels are added (#10757) Simo Lin 2025-09-22 16:23:07 -04:00
• 97c3823931 [router] refactor router and worker management 3/n (#10727) Simo Lin 2025-09-22 15:17:50 -04:00
• 60dbbd086a bugfix: Fix get_worker_urls_for_model in http/router.rs (#10754) Chang Su 2025-09-22 11:10:31 -07:00
• aa1c5cf5bd Add warnings and remove dependency for deterministic inference (#10724) Baizhou Zhang 2025-09-22 10:56:02 -07:00
• 592caab66a [Docs, minor] Fix LLM doc matrix (#10753) Adarsh Shirawalmath 2025-09-22 22:59:55 +05:30
• 2101d93b4f Fix CI TestChunkedSGMV (#10737) Lifu Huang 2025-09-22 01:09:58 -07:00
• 70e4b21853 Fix flaky logprobs test (#10728) Shangming Cai 2025-09-22 15:46:26 +08:00
• 944f1ea0ec fix capture_bs when speculative decoding enabled (#10730) feng397 2025-09-22 15:43:12 +08:00
• 9d7e82a0ab EPLB: prefer to use physical experts in the same node (#9849) Yingchun Lai 2025-09-22 15:34:30 +08:00
• f05805515b Convert FLASHINFER_WORKSPACE_SIZE to integer (#10731) Yang Yu 2025-09-22 15:26:42 +08:00
• 635ccda673 [4/4] Introduce CachedKernel to reduce CSGMV kernel launch overheads by 60% (#10709) Lifu Huang 2025-09-21 22:26:42 -07:00
• 1c3dbad8fe [Ascend] codeowner updates for ascend related files (#10699) ronnie_zheng 2025-09-22 05:47:01 +03:00
• e2ac7888b8 [2/2] Support deterministic inference for temperature > 0 (#10678) Qiaolin Yu 2025-09-21 19:36:08 -07:00
• 86527a4799 [deterministic inference] Move batch invariant pkg to sglang (#10695) Stefan He 2025-09-21 19:35:14 -07:00
• 134b4f7ec2 Support deterministic inference with triton backend (#10694) Ethan (Yusheng) Su 2025-09-21 18:20:40 -07:00
• f67d1f45bc [Auto Sync] Update deepseek_v2.py (20250922) (#10717) Yineng Zhang 2025-09-21 17:43:50 -07:00
• 0f04a5f428 Optimize cutlass int8 gemm kernel for large M on SM89 Ada GPU (#10714) Qi Yuhang 2025-09-22 08:04:27 +08:00
• 2f18602f13 fix: disable gpt-oss b200 ut (#10716) Yineng Zhang 2025-09-21 17:02:25 -07:00
• 56321e9fc2 [Router]fix: fix get_load missing api_key (#10385) Jimmy 2025-09-22 03:28:38 +08:00
• 12d6cf18f0 Refactors radix cache for extra key support (#10317) Xinyuan Tong 2025-09-21 11:16:16 -07:00
• fc3e542009 Update release-docs.yml (#10706) sglang-bot 2025-09-21 00:22:21 -07:00
• 08ecd0aa2a [3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592) Lifu Huang 2025-09-20 22:47:48 -07:00
• 720c1c8ca3 Super tiny fix extra logs (#10697) fzyzcjy 2025-09-21 12:30:54 +08:00
• d403c143e3 feat: update server args (#10696) Yineng Zhang 2025-09-20 18:52:19 -07:00
• cba0d8c309 [Feature] Support deterministic inference with FA3 backend (#10651) Stefan He 2025-09-20 17:50:21 -07:00
• f1d7892318 [Auto Sync] Update modelopt_quant.py (20250920) (#10688) Yineng Zhang 2025-09-20 02:37:49 -07:00
• 7c876de7f5 fix: remove awq_dequantize deps (#10686) Yineng Zhang 2025-09-20 01:47:01 -07:00
• ba94b82986 fix: update run_suite (#10685) Yineng Zhang 2025-09-20 01:22:06 -07:00
• 2b7417bf6a fix(disagg): fix sending KV cache in case of MLA for NIXL backend (#10673) dmitrygx 2025-09-20 10:30:10 +03:00
• f111649580 Replace os.environ in layernorm.py (#10684) Baizhou Zhang 2025-09-20 00:20:33 -07:00
• bd7eb0205a [Performance] Qwen3-Next: optimize causal_conv1d_fn triton kernel - up to 9% faster (#10680) Binyao Jiang 2025-09-20 00:12:43 -07:00
• 74cd6e3902 chore: upgrade mooncake 0.3.6.post1 to fix gb200 dockerfile (#10681) Shangming Cai 2025-09-20 15:12:26 +08:00
• b17e67df36 [Auto Sync] Update deepseek_v2.py (20250920) (#10683) Yineng Zhang 2025-09-19 23:43:31 -07:00
• 8ecef73f12 [1/2] Support deterministic inference with flashinfer attention backend (#10645) Baizhou Zhang 2025-09-19 23:34:29 -07:00
• 1d1ce62495 [router] refactor router and worker management 2.5/n (#10677) Simo Lin 2025-09-19 23:54:40 -04:00
• 60e2a7cead [Auto Sync] Update model_runner.py (20250920) (#10679) Yineng Zhang 2025-09-19 18:26:54 -07:00
• d88ef4a388 limit sgl-kernel causal conv1d to cuda only (#10648) Jinyan Chen 2025-09-20 07:59:37 +08:00
• 6f993e8b9e chore: cleanup docker image (#10671) Yineng Zhang 2025-09-19 16:56:49 -07:00
• 03ce92e594 router-spec: Reorder ChatCompletionRequest and fix validation logic (#10675) Chang Su 2025-09-19 16:41:21 -07:00
• 00eb5eb721 [router] refactor router and worker management 2/n (#10666) Simo Lin 2025-09-19 15:37:57 -04:00
• dab4663b4e [Auto Sync] Update .clang-format (20250919) (#10670) Yineng Zhang 2025-09-19 12:31:44 -07:00
• 610a6d6e86 fix: resolve sync issue (#10668) Yineng Zhang 2025-09-19 12:05:39 -07:00
• 36efd5be8a [router] refactor router and worker management 1/n (#10664) Simo Lin 2025-09-19 09:19:57 -04:00
• 68cdc1893d [router] preserve order of json params using preserve_order feature (#10661) Fabian Gebhart 2025-09-19 15:15:22 +02:00
• 7f399e4bce [HiCacheStorage]support page_first_direct layout for generic set&get (#10522) huangtingwei 2025-09-19 20:47:16 +08:00
• 873d858b28 [router] refactor worker to builder pattern 5/n (#10653) Simo Lin 2025-09-19 05:43:23 -04:00
• 3fa3c22ae2 Fix fast decode plan for flashinfer v0.4.0rc1 and upgrade sgl-kernel 0.3.11 (#10634) Baizhou Zhang 2025-09-19 01:25:29 -07:00
• 4f2055ad56 [router] refactor worker to builder pattern 4/n (#10650) Simo Lin 2025-09-19 02:49:10 -04:00
• 616a3e20df [sgl-kernel] Support moe_sum_reduce cuda kernel (#10321) Yuan Luo 2025-09-19 14:12:09 +08:00
• ac2a723bb3 [router] refactor worker to builder pattern 3/n (#10647) Simo Lin 2025-09-19 01:52:57 -04:00
• 56b991b12d [Feature]feat(get_ip): unify get_ip_xxx (#10081) Jimmy 2025-09-19 13:35:26 +08:00
• 780d6a22cd [router] refactor worker to builder pattern 2/n (#10633) Simo Lin 2025-09-19 00:47:56 -04:00
• 8b713c7248 Hicache L3 backend mooncake optimization configuration reading method (#10319) FlyPanda 2025-09-19 12:25:01 +08:00
• 5bfafdfcb4 chore: bump sgl-kernel 0.3.11 (#10630) Yineng Zhang 2025-09-18 18:43:20 -07:00
• 8c52de6fab feat: add fused moe config for Qwen3-Next-80B-A3B-Instruct on B200 (#10631) zixuanzhang226 2025-09-18 17:31:58 -07:00
• c1815a99b7 model support: Sarashina2VisionForCausalLM (#10632) Chang Su 2025-09-18 17:30:38 -07:00
• 4e6c4923a0 [Performance] Qwen3-Next: speed up update_mamba_state_after_mtp_verify by 10x; e2e up to 3.54% faster (#10586) Binyao Jiang 2025-09-18 17:13:59 -07:00
• b91cb67e7a [Performance] Qwen3-Next: replace arange to cached query_start_loc_li… (#10553) Binyao Jiang 2025-09-18 17:02:42 -07:00
• e7bc600304 [Feature] Speculative decoding support lookahead (#9873) Zhihao Zhang 2025-09-19 07:42:41 +08:00
• 2a2ff9a840 refactor: use registry for _get_attention_backend_from_str (#10629) Yineng Zhang 2025-09-18 16:27:59 -07:00
• 5291f32d75 [router] refactor worker to builder pattern 1/n (#10628) Simo Lin 2025-09-18 16:25:40 -04:00
• 67073dde85 Garbage collector regression in the online server (#10621) brayden-hai 2025-09-18 11:52:35 -07:00
• 9a5c42f9ad feat: Add FlexAttention Backend for Efficient Sparse Attention (#9947) yuk.igalaxy 2025-09-19 02:49:17 +08:00
• 388c05d544 Fix bias handling in TritonMoeQuantInfo within quantization/mxfp4.py (#10579) yhyang201 2025-09-19 02:44:43 +08:00
• fc809665fd [Performance] qwen3-next improve causal conv1d in prefill phase (#10595) Jinyan Chen 2025-09-19 02:42:49 +08:00
• 6fd4816d9f Fix sgl_kernel import failure on devices other than CUDA (#10610) Zaili Wang 2025-09-19 02:38:02 +08:00
• 1344ebc833 support qwen3-next-fp8 deepep (#10622) Yi Zhang 2025-09-19 02:36:22 +08:00
• e07b21ceaf update deepep version for qwen3-next deepep moe (#10624) Yi Zhang 2025-09-19 02:35:22 +08:00
• 52f248cd31 Feat/add heartbeat mechanism for nixl conn (#10222) shaharmor98 2025-09-18 20:20:42 +03:00
• 93f75778be [RL] Add destroy process group api (#9979) penguin_wwy 2025-09-19 00:31:56 +08:00
• 4039c626e2 fix deepep assert when PD disaggregation == null (#8274) AlphaBaby 2025-09-19 00:31:01 +08:00
• db71c38fcd Scale kkt after reduction (#10604) Yi Zhang 2025-09-18 20:51:40 +08:00
• 7a68b4225a [improvement] add average input/output token length for hicache benchmark stats output (#10525) zhannngchen 2025-09-18 15:38:03 +08:00
• 60fc5b51f6 chore: upgrade mooncake 0.3.6 (#10596) Shangming Cai 2025-09-18 15:19:30 +08:00
• a13dd1e492 [PD] Improve disaggregation common backend and refactor mooncake backend (#10273) Shangming Cai 2025-09-18 13:58:07 +08:00
• d500eb9173 aiter v0.1.5.post2 (#10563) HAI 2025-09-17 22:10:45 -07:00
• 1ccd59c715 [HICache] introduce evict policy (#10190) Xuchun Shang 2025-09-18 11:10:20 +08:00
• c32fb7a24d [ROCm] Fix fp8 quantization accuracy issue. (#10558) sogalin 2025-09-18 08:44:59 +08:00
• 1ba137e98f Enable trtllm mla prefix extend (#10526) Shu Wang 2025-09-17 18:44:11 -05:00
• de28f8e741 vlm: remove redundant d2h movement of mm feature tensors (#9987) Kevin Xiang Li 2025-09-17 15:00:39 -07:00
• 564050766d fix: update dsv3 fp4 ut (#10584) Yineng Zhang 2025-09-17 14:34:58 -07:00
• b73ac629cd [BugFix] Fix incorrect hidden_states_tensor in pd disaggregation + eagle (#9976) Ziming Huang 2025-09-18 01:37:14 +08:00
• 77098aea7b [HiCache] Add tests for hicache storage mooncake backend (#10171) Teng Ma 2025-09-18 01:07:16 +08:00
• 5ccf0b03bd [bench] Fix random seed in bench_one_batch_server (#10548) Liangsheng Yin 2025-09-17 19:30:32 +08:00
• a77564e0fb CUDA Arch Independent (#8813) EduardDurech 2025-09-17 08:01:45 +02:00
• 4f9e71df3c Remove duplicated code (#10545) Yichen Yan 2025-09-17 11:48:22 +08:00
• 541551cefe [bugfix]hicache bench_long_context.py run failed (#10523) zhannngchen 2025-09-17 11:27:06 +08:00
• 124097fc5b enable prefix cache with dp (#10459) Shu Wang 2025-09-16 20:26:58 -05:00
• e1d45bc280 Fix decord dependency for aarch64 docker build (#10529) kyleliang-nv 2025-09-16 17:34:37 -07:00
• 14fdd52740 feat: add priority based scheduling with priority based request acceptance and preemption (#8746) harrisonlimh 2025-09-16 17:10:10 -07:00
• f949ad5794 [Auto Sync] Update activation.py, chunk_cache.py, utils.py (20250917) (#10538) Lianmin Zheng 2025-09-16 17:06:43 -07:00
• c49484a658 [Auto Sync] Update scheduler_profiler_mixin.py, rpd_utils.p... (20250916) (#10494) Lianmin Zheng 2025-09-16 17:02:20 -07:00
• a2f7218a2e support using fa4 on deepseek on blackwell (#9928) cicirori 2025-09-17 07:16:06 +08:00
• 311de47bb7 [2/2] Speed up trtllm_mla attention backend (#10474) fzyzcjy 2025-09-17 06:49:22 +08:00
• 373080ea6c skip vision_model for lora (#10530) gongwei-130 2025-09-16 12:34:42 -07:00
• 7f028b07c4 Fix formatting in long code blocks (#10528) Philip Kiely - Baseten 2025-09-16 12:02:05 -07:00
• 0abb41c70d adjust import setuptools_rust (#10524) ybyang 2025-09-16 23:01:58 +08:00
• 925dbb3218 [CPU] fix CPU backend sel. issue for Llama4 (#10511) Zaili Wang 2025-09-16 17:57:45 +08:00
• 8df7353af3 Support sgl-router parallel_batch in bench_one_batch_server (#10506) fzyzcjy 2025-09-16 17:52:57 +08:00