Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• ec15c8360e Optimize Qwen3-moe model by using flashinfer fused allreduce (#9973) Yuan Luo 2025-09-04 20:48:53 +08:00
• 106c2b31fb feat(hicache): Add generic hicache ci e2e test and benchmark test (#9846) hzh0425 2025-09-04 20:43:46 +08:00
• c67569491c Ensure chunked request extension length respects both rem_chunk_tokens and rem_total_tokens limits (#10003) pansicheng 2025-09-04 19:15:26 +08:00
• 27e8ffed37 [1/N] DP-refactor: move dp balance code into scheduler's mixin class (#10004) Liangsheng Yin 2025-09-04 16:53:58 +08:00
• 4dbb34fe43 fix: health_generate endpoint in mini_lb (#9997) wxsm 2025-09-04 16:52:28 +08:00
• 1e18a341e9 [Bugfix] fix pd chat completion protocol for batching support (#10016) Tony Lu 2025-09-04 16:43:16 +08:00
• 2c562fd2d0 Fix Llama 4 with MXFP4 dynamic quant on MI35x (#9993) Hubert Lu 2025-09-04 00:48:58 -07:00
• b648d86216 [Fix] gpt-oss mxfp4 model run failed on ROCm platform (#9994) kk 2025-09-04 13:34:17 +08:00
• bbf261ae4a [router] fix grpc connection mode detection (#9999) Simo Lin 2025-09-04 00:36:16 -04:00
• 4f8a982d52 [router] clean up dependency injector to use ctx (#10000) Simo Lin 2025-09-04 00:35:51 -04:00
• d966b902af [router] move tokenizer, reasoning, tool initialization to server (#9996) Simo Lin 2025-09-03 22:35:13 -04:00
• de9217334b feat: add gpt oss b200 ci (#9988) Yineng Zhang 2025-09-03 17:26:38 -07:00
• 397448ebbc [Auto Sync] Update parallel_state.py, few_shot_gsm8k.py (20250903) (#9986) Lianmin Zheng 2025-09-03 16:55:43 -07:00
• 66d5d0425c Minor update regarding issue #9704 (#9733) Elfie Guo 2025-09-03 16:52:07 -07:00
• 73179b764a nsys profile output kernel classifier (#9314) Grace Ho 2025-09-03 16:22:33 -07:00
• 8cbf71dc2d Triton 3.4.0 MoE config for Deepseek TP16 H100 (#9978) Szymon Ożóg 2025-09-03 22:16:16 +02:00
• 56eb5d0a3d fix swa clear(): rename is_in_free_group to is_not_in_free_group (#9914) Xinyuan Tong 2025-09-03 18:42:12 +00:00
• 4ed9053ecf Remove mrope position sync (#9460) timmy-feng 2025-09-03 11:40:53 -07:00
• 5e19b159b0 [router] add chat_template_kwargs in ChatCompletionRequest (#9958) Tony Lu 2025-09-04 01:43:52 +08:00
• 788b19a532 [router] Add Rerank API Specification (#9906) Frank Fang 2025-09-03 23:30:29 +08:00
• f78b7fd16d [1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel) (#9953) Yuhao Yao 2025-09-03 18:28:27 +08:00
• b1fb7e458c [benchmark] add flashinfer_allreduce_fusion benchmark (#9937) Xiaoyu Zhang 2025-09-03 16:31:01 +08:00
• 1b2ff4fb7f Revert "Optimized deepseek-v3/r1 model performance on mxfp4 run (#9671)" (#9959) Yineng Zhang 2025-09-03 00:50:04 -07:00
• 2c7ca33abb Revert "[Fix] DeepSeek EP accuracy issue on B200 GPUs (#9946)" (#9955) Yineng Zhang 2025-09-02 23:49:56 -07:00
• df397a72e8 [feat] Add P/D attention select for draft model (#9755) Ximingwang-09 2025-09-03 13:47:23 +08:00
• 5dfcd6c207 add proctitle for tokenizers (#9952) Liangsheng Yin 2025-09-03 13:31:38 +08:00
• 0dfd54d11d Optimized deepseek-v3/r1 model performance on mxfp4 run (#9671) kk 2025-09-03 13:26:28 +08:00
• bcbeed714f Qwen FP8/NVFP4 ModelOPT Quantization support (#7912) jingyu-ml 2025-09-02 22:56:03 -05:00
• cc9a31c662 Update tool_chat_template_deepseekv31.jinja (#9895) 南京小汤包 2025-09-03 11:29:21 +08:00
• d631290e32 Remove annoying warnings in sgl kernel build (#9905) Lianmin Zheng 2025-09-02 20:18:25 -07:00
• 37565b7f21 fix(cache): move ongoing_prefetch pop after validation to prevent leak (#9927) JinYan Su 2025-09-03 10:39:34 +08:00
• 6243c36702 [Fix] DeepSeek EP accuracy issue on B200 GPUs (#9946) Al-Ekram Elahee Hridoy 2025-09-02 20:31:15 -06:00
• 60e37f8028 Move parsers under a single folder (#9912) Lianmin Zheng 2025-09-02 18:25:04 -07:00
• 369b143366 [HiCache] Minor fix on file storage backend (#9869) Zhiqiang Xie 2025-09-02 15:52:37 -07:00
• 03dbf1aa8e [model] support MiniCPM-V 4.0 (#8747) tc-mb 2025-09-03 06:33:03 +08:00
• 11dcabc545 Grpc client (#9939) Chang Su 2025-09-02 11:47:35 -07:00
• 4d89389c4f Fix the key passing issue in page first layout. (#9929) hzh0425 2025-09-03 02:30:11 +08:00
• 9491d6e554 [router] include rust benchamrks (#9932) Simo Lin 2025-09-02 09:32:09 -07:00
• f64b8e3e4e Support the internvl3.5 family models in sglang (#9705) yilian49 2025-09-02 08:06:48 -06:00
• 53976fce97 [Hicache] Generic page get bugfix (#9909) ykwd 2025-09-02 20:22:06 +08:00
• 18f91eb639 chore: bump v0.5.2rc1 (#9920) Yineng Zhang 2025-09-02 04:43:34 -07:00
• 8766b3aca8 fix: update router deps (#9921) Yineng Zhang 2025-09-02 03:28:58 -07:00
• 1db649ac02 [feat] apply deep_gemm compile_mode to skip launch (#9879) JieXin Liang 2025-09-02 18:20:30 +08:00
• a1e5d78115 fix parallel_state.py current_platform bug (#9919) Xiaoyu Zhang 2025-09-02 18:17:15 +08:00
• b7361cc444 [Fix] fix the issue encountered when inference LongCat-Flash/MTP EP MoE on b200 (#9916) Guoyuan Lin 2025-09-02 18:11:14 +08:00
• a96c5b5c14 chore: bump v0.3.8 sgl-kernel (#9907) Yineng Zhang 2025-09-02 01:27:26 -07:00
• b9eb0d9c2b Change tensor alignment method to mn major (#9844) Mohammad Miadh Angkad 2025-09-02 16:23:13 +08:00
• 1fbfdebe6b [chore] fix dead links in doc (#9913) Lifu Huang 2025-09-02 00:28:26 -07:00
• a25e8e42eb Move multi-tokenizer event loop to better place (#9902) Shangming Cai 2025-09-02 14:12:21 +08:00
• d4a938417d [feat] Support tp mode for DeepSeek-R1-W4AFP8 (#8118) chenxj 2025-09-02 13:17:26 +08:00
• 21e1bc475c [router] fix FunctionCallResponse proto, support arguments is null (#9875) Bruce-x-1997 2025-09-02 11:37:15 +08:00
• 9a0cac1be0 [router] add grpc pd and regular router init (#9893) Chang Su 2025-09-01 20:06:15 -07:00
• b5245064f6 [code style] restruct fused_moe to avoid very long single file (#9878) Xiaoyu Zhang 2025-09-02 11:04:27 +08:00
• 9d9fa9a537 [router] Fix short timeout for the prefill client (#9803) LukasBluebaum 2025-09-02 04:57:04 +02:00
• 58d06fdc95 [HiCacheStorage]: Improve 3fs kvstore‘s performance and resolve mla issues (#9876) hzh0425 2025-09-02 10:01:48 +08:00
• cb9e0e4180 [HiCacheStorage] fix abort request host memory leaks (#9874) huangtingwei 2025-09-02 09:59:29 +08:00
• 9db8025376 support fp8 kvcache for hybrid attn backend on GPT-OSS (#9783) Rain Jiang 2025-09-01 12:17:12 -07:00
• 598c0bc19d [router] add tokenizer download support from hf hub (#9882) Chang Su 2025-09-01 10:40:37 -07:00
• b361750a4a Mooncake store get zero copy meta optimization (#9857) huangtingwei 2025-09-01 18:27:56 +08:00
• 16e56ea693 chore: bump v0.5.2rc0 (#9862) Yineng Zhang 2025-09-01 03:07:36 -07:00
• 349b491c63 chore: upgrade flashinfer 0.3.0 (#9864) Yineng Zhang 2025-09-01 03:07:19 -07:00
• 5f77e1292d Support Multi Process Tokenizer Manager(#6555) (#8964) ybyang 2025-09-01 16:00:13 +08:00
• 4750cddf68 Update docker build workflows for gfx942 ROCm 7.0. (#9794) Sai Enduri 2025-09-01 00:37:12 -07:00
• 065e523d7b Tiny allow DeepGEMM on cu12.9 (#9858) fzyzcjy 2025-09-01 14:29:56 +08:00
• 7de2ce45b2 Disable radix cache in test_lora_update.py for better stability (#9852) Baizhou Zhang 2025-08-31 22:28:22 -07:00
• 8c2ffaaf0f fix(hicahce-long-bench): adjust context workload generator to use full query set (#9847) hzh0425 2025-09-01 05:51:18 +08:00
• 20445327b2 fix inconsistent arguments for generated shared prefix bench (#9073) Pawel Kowalski 2025-08-31 23:27:33 +02:00
• 6d3c20cf5b fix set_interal_state API (#9850) Liangsheng Yin 2025-09-01 01:31:35 +08:00
• 8b6966d020 [HiCache] Storage Refactoring (#9797) Zhiqiang Xie 2025-08-31 07:58:21 -07:00
• a391f73adc Fuse gate_proj and up_proj in Qwen 2.5 VL's vision MLP (#9661) Kevin Xiang Li 2025-08-31 04:08:28 -07:00
• 25c7395934 Fix input logprob index (#9841) Lianmin Zheng 2025-08-31 02:56:47 -07:00
• f05c68733e [HiCache] Clear kvcache in storage backend with fastAPI (#9750) Teng Ma 2025-08-31 17:41:44 +08:00
• 9a0d0b754d [Performance] Improve Qwen RMSNorm by replacing with native RMSNorm op (#9709) Vincent Zhong 2025-08-31 05:20:50 -04:00
• ba861293cf [feat]Ascend NPU Gemma-3-12b and Gemma-3-27b support (#8909) VDV1985 2025-08-31 10:25:07 +03:00
• c112bcc461 [router] global tool parser registry (#9840) Chang Su 2025-08-30 23:35:39 -07:00
• 5e194b2143 [Model] Support Meituan LongCat-Flash && LongCat-Flash-MTP (#9824) Guoyuan Lin 2025-08-31 14:29:21 +08:00
• fd5ce576a4 Tool parser.benchmark (#9835) Chang Su 2025-08-30 21:08:11 -07:00
• 92d79646e5 [router] add reasoning parser readme (#9837) Simo Lin 2025-08-30 21:06:23 -07:00
• f9076a5a2c hot fix for mooncake batch set api (#9836) Zhiqiang Xie 2025-08-30 21:01:51 -07:00
• 646076b71e Update guidelines for syncing code between repos (#9831) Lianmin Zheng 2025-08-30 16:10:35 -07:00
• 0d04008936 [CI] Code sync tools (#9830) Lianmin Zheng 2025-08-30 16:02:29 -07:00
• 05e4787243 [CI] Fix the trigger condition for PR test workflows (#9761) Lianmin Zheng 2025-08-30 15:47:10 -07:00
• 1e61b4960f [Auto Sync] Update parallel_state.py (20250830) (#9828) Lianmin Zheng 2025-08-30 14:25:39 -07:00
• 300676afac chore: upgrade transformers 4.56.0 (#9827) Yineng Zhang 2025-08-30 14:07:34 -07:00
• 7fe89f7cdb [sgl-kernel] fix: fix missing FetchContent_Populate for fmt (#9826) PGFLMG 2025-08-31 03:57:42 +08:00
• 9970e3bf32 chore: upgrade sgl-kernel 0.3.7.post1 with deepgemm fix (#9822) Yineng Zhang 2025-08-30 04:02:25 -07:00
• 70eedb58bb Fix typo in warning message about DeepGEMM JIT (#9802) Mohammad Miadh Angkad 2025-08-30 18:35:53 +08:00
• 9c99949ef3 chore: update Dockerfile (#9820) Yineng Zhang 2025-08-30 03:08:14 -07:00
• c5082f0f73 chore: fix cuda driver api issue and bump sgl-kernel 0.3.7.post1 (#9746) Yineng Zhang 2025-08-30 02:01:54 -07:00
• 836873b99f Fix memory leak when aborting decode request in PD-Disagg (#9817) Liangsheng Yin 2025-08-30 14:36:03 +08:00
• 8abe8deae6 fix: dsv3 lite q_lora_rank none (#9815) Yineng Zhang 2025-08-29 23:24:14 -07:00
• 1e85589dc5 Make fp4_quantize kernels work on sm103 (#9807) hlu1 2025-08-29 21:15:08 -07:00
• c2a26e725c feature(eplb): add min-rebalancing-utilization-threshold for eplb (#8345) hzh0425 2025-08-30 11:24:29 +08:00
• 591e6c5983 Small bug fix in transformers model implementation (#9809) yilian49 2025-08-29 19:51:44 -06:00
• 42f34437ab Adds initialize_moe_config to bench_one_batch so MOE backend is respected (#9670) pranavm-nvidia 2025-08-29 17:29:32 -07:00
• 5c34b4f1c7 [NVIDIA] [2/N] Optimize silu_and_mul_scaled_fp4_grouped_quant perf (#9556) Kaixi Hou 2025-08-29 17:17:03 -07:00
• ff9b561817 Fix TRTLLM MLA Cuda KV Blocks Causing accuracy drop (#9675) Faraz 2025-08-29 20:16:10 -04:00
• fcd72bd100 [ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization (#9712) Pavani Majety 2025-08-29 17:13:52 -07:00
• 3d8fc43400 chore: upgrade flashinfer 0.3.0rc1 (#9793) Yineng Zhang 2025-08-29 16:24:17 -07:00
• 87a0f7d2c2 [feat] Support EAGLE3 for Qwen2 (#9216) KerwinKai 2025-08-30 03:59:51 +08:00