Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 6ce0ed073b Apply constraint grammar to EAGLE (#6499) Ke Bao 2025-05-22 08:18:41 +08:00
• 969660c762 Recover from corrupted cache file in bench serving (#6510) fzyzcjy 2025-05-22 08:13:54 +08:00
• 16d4f6801b doc: Update README.md with adding deepwiki badge to enable weekly auto-refresh (#6508) Xinyuan Tong 2025-05-21 16:27:34 -07:00
• ada268fd05 fix: EXAONE when using tie_word_embeddings (#5759) Kyungmin Lee 2025-05-22 03:30:04 +09:00
• cfe48c5902 [CPU] Fix build issue (#6419) blzheng 2025-05-22 02:17:10 +08:00
• d4c038daed [Fix]Fix capture fail bug for DeepSeek (#6275) Baizhou Zhang 2025-05-21 11:11:20 -07:00
• 55f6005f53 Fix bench_one_batch_server (#6503) fzyzcjy 2025-05-22 02:08:17 +08:00
• 7222e1dacc Let bench_one_batch_server use sharegpt data to make expert distribution more natural (#5573) fzyzcjy 2025-05-21 17:08:43 +08:00
• 505eec4dc9 Tiny make Lint CI show diff (#6445) fzyzcjy 2025-05-21 17:06:25 +08:00
• ccfe5c009d Support redundant experts in expert parallel (#6461) fzyzcjy 2025-05-21 17:05:53 +08:00
• a071dc4084 Tiny add stage assertions to DeepEPDispatcher to avoid misuse (#6467) fzyzcjy 2025-05-21 17:05:05 +08:00
• a40aecc5a3 Fix num_qps_per_rank computation when providing custom DeepEP configuration (#6468) fzyzcjy 2025-05-21 17:04:33 +08:00
• d6e1d28c8a Refactor DeepSeek attention dispatching (#6476) fzyzcjy 2025-05-21 17:03:39 +08:00
• 7c347259ff [RL] allow weight updation with dp attention enabled (#6311) Zilin Zhu 2025-05-21 16:58:55 +08:00
• 669caa0a3f [router] support http2 in router (#6487) Zilin Zhu 2025-05-21 16:42:45 +08:00
• 4024e1d2a8 Implement Siglip Vision model, and support BNB quantization for gemma3-mm (#5339) Jiajun Li 2025-05-20 23:53:46 -07:00
• 5c0b38f369 aiter attention-backend (default enabled on AMD/ROCm) (#6381) HAI 2025-05-20 22:52:41 -07:00
• 30ca18f423 Refactor group_concurrent_contiguous in NIXL (#6214) Yuan Luo 2025-05-21 11:55:04 +08:00
• 03886917bd Disable all two stream overlap on amd (#6475) Lianmin Zheng 2025-05-20 19:06:59 -07:00
• 66324895c6 [docs] Fix torch version (#6472) Wenxuan Tan 2025-05-20 12:53:14 -05:00
• 13feffd082 Fix master CI for DeepSeek (#6447) fzyzcjy 2025-05-20 15:31:42 +08:00
• e98afbe042 Support dispatching logical to physical experts (#6385) fzyzcjy 2025-05-20 13:13:55 +08:00
• 69af3ec35f [doc] add note for get_num_kv_splits in triton_backend (#6444) JieXin Liang 2025-05-20 12:40:21 +08:00
• 32cc66efa5 Update extend/decode attention kernel for CPU in sgl-kernel and add UTs (#6405) YanbingJiang 2025-05-20 12:23:17 +08:00
• 83f2d9d4ed [QuickFix] fix gptq model initialize (#6429) PGFLMG 2025-05-20 12:17:10 +08:00
• 6317c5c61f Address performance regression: disable multiple streams on ROCm (#6412) HAI 2025-05-19 21:16:20 -07:00
• cba1cdbc46 Support DeepSeek EPLB algorithm with static distributions (#6387) fzyzcjy 2025-05-20 12:06:21 +08:00
• c471d39eb9 Support loading weights when physical experts are different from logical experts (#6386) fzyzcjy 2025-05-20 12:05:53 +08:00
• d0443275f0 Refactor DeepSeek logic into atomic operations (#6326) fzyzcjy 2025-05-20 12:05:30 +08:00
• 17d080b7ae Remove Cargo.lock, add it into .gitignore (#6438) Liangsheng Yin 2025-05-20 12:01:32 +08:00
• 1b19df4b2a Refactor communication logic of DeepSeek for extensibility and understandability (#6321) fzyzcjy 2025-05-20 11:14:48 +08:00
• f0653886a5 Expert distribution recording without overhead for EPLB (#4957) fzyzcjy 2025-05-20 11:07:43 +08:00
• b146555749 Revert "Implement return_hidden_states for the OpenAI API (#6137)" (#6440) Yineng Zhang 2025-05-19 18:21:29 -07:00
• b06215daed [BUG] fix stop_profile crash (#6431) Yi Zhang 2025-05-20 08:30:33 +08:00
• 7adf245ba2 [Metrics] Add KV events publishing (#6098) Trevor Morris 2025-05-19 14:19:54 -07:00
• 299fd22f9e Fix throughput threshold for amd ci test (#6414) Baizhou Zhang 2025-05-19 14:17:41 -07:00
• 506e5de8fe Improve supported models doc (#6430) simveit 2025-05-19 19:43:35 +02:00
• 844e2f227a Fix nodeepgemm init (#6417) lukec 2025-05-19 15:44:03 +08:00
• 4f39bcf7ab Implement return_hidden_states for the OpenAI API (#6137) kyle-pena-kuzco 2025-05-19 01:30:25 -04:00
• 31c9569bb8 Fix request id error (#6401) fzyzcjy 2025-05-19 09:58:59 +08:00
• 1be6956d1b [Bugfix] Fix field error in v1_embedding_request (#6400) Chang Su 2025-05-18 15:58:29 -07:00
• 626ccb7d3f vlm: tensor hash kernel (#5974) Mick 2025-05-19 06:38:16 +08:00
• 72bfb0baf0 Refactor DeepSeek MoE layer to unify the two forward branches (#6325) fzyzcjy 2025-05-19 06:34:36 +08:00
• 155214952b refactor: Extract repeated member variables in KVCache subclasses to base class. (#6323) wangxiyu191 2025-05-19 06:28:15 +08:00
• ebe58d545d [Misc] Implement RankZeroFilter for rank-specific logging in model_runner.py (#6333) Chang Su 2025-05-18 15:27:13 -07:00
• 066cf44546 [OAI] Add rid tracing for v1/embeddings and fix rid type in Chat (#6397) Chang Su 2025-05-18 13:05:38 -07:00
• 6dc6b30637 Add missing model to doc (#6396) applesaucethebun 2025-05-18 15:57:58 -04:00
• 1f30c05d4a [fix] fix fa3 forward_decode with spec_decode (#6395) JieXin Liang 2025-05-19 03:50:15 +08:00
• 5dd62c3a6f Add fp8 shared_expert kernel for CPU in sgl-kernel and add UT (#6339) Chunyuan WU 2025-05-19 03:42:15 +08:00
• f11481b921 Add 4-GPU runner tests and split existing tests (#6383) fzyzcjy 2025-05-19 02:56:51 +08:00
• 9d24c3ffb0 chore: tiny remove duplicated code (#6392) doujiang24 2025-05-18 17:17:32 +08:00
• 24161c5913 The Gemma template is missing a newline after the user role. (#6331) Yury Sulsky 2025-05-18 01:57:27 -07:00
• eabcf82acb feat: add long context example (#6391) Yineng Zhang 2025-05-18 01:45:17 -07:00
• c47a51db7e Clean up AMD CI (#6365) Sai Enduri 2025-05-18 01:17:28 -07:00
• 11553c1a37 Add pipeline parallelism for Qwen2 and Qwen3 Model (#6250) libra 2025-05-18 15:42:55 +08:00
• 01dd39bac1 refactor: minor refactors regarding multimodal processing (#6187) Mick 2025-05-18 13:53:20 +08:00
• b3f3d610fd Do not use FA3 for mistral (#6379) Lianmin Zheng 2025-05-17 19:47:34 -07:00
• f07c6a009b chore: upgrade sgl-kernel v0.1.3 (#6377) Yineng Zhang 2025-05-17 19:47:05 -07:00
• 4bb816d444 Fix CI tests (#6362) Lianmin Zheng 2025-05-17 19:16:45 -07:00
• c250939ecb [Fix Chat API] add request id for chat/completion for tracing (#6364) ybyang 2025-05-18 09:58:22 +08:00
• b6909aa223 fix: allow launch_dummy_health_check_server to start inside of running asyncio loop (#6330) ishandhanani 2025-05-17 18:32:41 -07:00
• f87283573e Add expert distribution APIs for engine (#6290) fzyzcjy 2025-05-18 09:31:51 +08:00
• 73187152a4 Reland tiny refactor DefaultModelLoader.Source (#6041) fzyzcjy 2025-05-18 08:11:20 +08:00
• 4086566516 Fix expert distribution recorder and profiler command stuck forever (#6284) fzyzcjy 2025-05-18 08:10:44 +08:00
• fd08c04821 Support custom DeepEP tuning config (#6257) fzyzcjy 2025-05-18 08:09:42 +08:00
• 26ebb849eb Tiny refactor bench_serving to extract RequestFuncOutput.init_new (#6108) fzyzcjy 2025-05-18 08:08:52 +08:00
• 02973cd9a4 Tiny refactor bench_serving to improve extensibility (#6134) fzyzcjy 2025-05-18 08:07:58 +08:00
• 6d95a35abf Support outputing details for bench_serving (#6107) fzyzcjy 2025-05-18 08:06:52 +08:00
• 01d2838c0f Fix stop_profile does not wait for finishing (#4741) fzyzcjy 2025-05-18 08:06:15 +08:00
• e3b8a72291 [fix] illegal memory in _fwd_kernel_ep_scatter_2 and _fwd_kernel_ep_gather (#6348) xutizhou 2025-05-18 08:01:42 +08:00
• 3cf1473a09 Use monotonic clock for interval measurement (#6211) Lifu Huang 2025-05-17 16:49:18 -07:00
• 2716830802 Speed up when having padding tokens in DeepEP (#6175) fzyzcjy 2025-05-18 07:44:05 +08:00
• e3bed74afb [router] Add /list_workers endpoint to router (#6366) Zilin Zhu 2025-05-18 00:49:02 +08:00
• e9ef39d2e9 docs: Update the MD files (#6373) Vincent Zhong 2025-05-17 12:23:16 -04:00
• 205d5cb407 perf: Optimize local attention memory allocation in FlashAttentionBackend (#6356) Chang Su 2025-05-17 01:45:46 -07:00
• 3d7f7a43c8 chore: bump sgl-kernel v0.1.3 (#6368) Yineng Zhang 2025-05-17 00:15:55 -07:00
• 2df9d40aa6 Minor code cleanup refactor for DeepSeek models (#6324) fzyzcjy 2025-05-17 10:06:03 +08:00
• 8dc191f237 Fix one wasted kernel in DeepSeek and minor refactor (#6316) fzyzcjy 2025-05-17 10:05:33 +08:00
• 64825b8395 model(vlm): mistral 3.1 (#5099) Kiv Chen 2025-05-16 18:36:18 -07:00
• 69748d088d docs: update readme (#6361) Yineng Zhang 2025-05-16 16:16:23 -07:00
• dcc0a45618 Fix amd ci (#6360) Lianmin Zheng 2025-05-16 15:33:10 -07:00
• c2b7ddca49 [Minor] cleanup unused imports (#6358) Lianmin Zheng 2025-05-16 14:52:38 -07:00
• abebd9399c Update CODEOWNERS (#6359) Lianmin Zheng 2025-05-16 14:51:36 -07:00
• 4bd2952a37 feat: add dp attention support for Qwen 2/3 MoE models, fixes #6088 (#6121) Fr4nk1in 2025-05-17 05:44:10 +08:00
• 6fc9357503 [2/2] Add python wrapper for CUTLASS FP8 Blockscale MoE Kernel. (#5694) Elfie Guo 2025-05-16 13:14:07 -07:00
• 839fb31e5f [Fix] Improve dependencies for Blackwell image (#6334) Baizhou Zhang 2025-05-16 12:38:22 -07:00
• f19a9204cd Support precomputed multimodal features for Qwen-VL and Gemma3 models. (#6136) Yury Sulsky 2025-05-16 12:26:15 -07:00
• c23a7072b6 Upgrade CUTLASS 4.0 (#6336) Elfie Guo 2025-05-15 17:42:23 -07:00
• e07a6977e7 Minor improvements of TokenizerManager / health check (#6327) Lianmin Zheng 2025-05-15 15:29:25 -07:00
• cd8d4b9dfc Fix lora bench (#6302) Qiaolin Yu 2025-05-15 13:09:55 -04:00
• f194e14fb7 Reduce MoE memory usage (#6147) fzyzcjy 2025-05-16 00:38:28 +08:00
• cfc9f9ab8d Fix gpu mem check on CPU (#6317) Yi Liu 2025-05-16 00:37:45 +08:00
• fb4959b2c5 Add fp8 gemm kernel for CPU in sgl-kernel and add gemm UT (#6216) Chunyuan WU 2025-05-16 00:10:40 +08:00
• 9a405274e2 [misc] remove redundant platform codes (#6298) JieXin Liang 2025-05-15 15:51:30 +08:00
• 2e4babdb0a [Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) quinnrong94 2025-05-15 15:48:09 +08:00
• 44a3783d13 [fix][RL] Remove the incorrect barrier in init_weights_update_group (#5914) Zilin Zhu 2025-05-15 10:15:21 +08:00
• f3bf611054 feat: add flush cache to EngineBase and HttpServerEngineAdapter (#6009) Junrong Lin 2025-05-15 10:15:02 +08:00
• 198b9056d1 [AMD] Fix Llama 4 Scout and Maverick accuracy issues on MI300X (#6274) Hubert Lu 2025-05-14 15:07:29 -07:00
• 73eb67c087 Enable unit tests for AMD CI. (#6283) Sai Enduri 2025-05-14 12:55:36 -07:00
• 9a91fa0ed1 docs: fix a bad redirect (#6300) Brayden Zhong 2025-05-14 13:27:19 -04:00