Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 8879944800 ROCm/AITER CK_MoE: update 2-stage kernels & support both Activations (#5228) HAI 2025-04-10 18:19:57 -07:00
• a879811c4b Fix torch.compile cacheing (#5259) Richard Zou 2025-04-10 21:08:45 -04:00
• a222945df2 Update Makefile / build script to avoid installing incompatible torch dependency (#5245) Elfie Guo 2025-04-10 15:21:02 -07:00
• ed01b4515e [Misc] Clean sgl-kernel test (#5216) PGFLMG 2025-04-11 02:28:41 +08:00
• d050df368c ROCm sgl-kernel: compatible to later torch (#5167) HAI 2025-04-10 09:18:36 -07:00
• 76f44c2a8d Fix deepseek-v3 with torch.compile in PyTorch 2.6. (#5213) Richard Zou 2025-04-10 12:14:38 -04:00
• 1078396f47 Update deps for mllama4 (#5215) Ke Bao 2025-04-11 00:12:44 +08:00
• 7e4f72dd8c [PD] Add get_contiguous_buf_infos interface for MLATokenToKVPool (#5204) Teng Ma 2025-04-10 20:05:34 +08:00
• 4c31ae9f6d [PD] Support KV transfer with mooncake (#4880) Teng Ma 2025-04-10 14:23:23 +08:00
• f730362ee2 reduce moe_align_block_size_kernel small batch mode overhead (#5086) Xiaoyu Zhang 2025-04-10 08:59:35 +08:00
• e3c4bd3153 Fix DeepSeek error when using DeepEP mode (#5190) fzyzcjy 2025-04-10 08:43:22 +08:00
• 5db37c8626 [metrics] Add in queue metrics (#4444) Stefan He 2025-04-09 17:19:27 -07:00
• 4cb53ecd0c fix: log warning when disable cuda graph (#5209) Yineng Zhang 2025-04-09 14:16:13 -07:00
• 456b008bd8 Add H20 dtype fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5196) Zhaoyang Hao 2025-04-10 02:54:36 +08:00
• ebf495f013 sgl-kernel use cutlass latest version for fp8 blockwise gemm (#5207) Yi Zhang 2025-04-10 02:47:04 +08:00
• 7f875f1293 update grok test (#5171) saienduri 2025-04-09 11:09:47 -07:00
• fbebcb7aa4 model: support mllama4 (#5144) Mick 2025-04-10 00:28:44 +08:00
• 87eddedfa2 [ci] fix ci test fused_moe op (#5102) Xiaoyu Zhang 2025-04-09 23:52:46 +08:00
• 4065248214 Support Llama4 fp8 inference (#5194) HandH1998 2025-04-09 20:14:34 +08:00
• 86a876d883 Optimize topk operation in llama4 (#5128) fzyzcjy 2025-04-09 17:50:22 +08:00
• 92823069c4 Fix ci test "test_eval_fp8_accuracy" failed (#5185) kk 2025-04-09 17:44:05 +08:00
• d2e507df3c [Misc] clean up vllm in sgl-kernel test (#5189) yinfan98 2025-04-09 16:22:13 +08:00
• 61970b08d8 Let bench_one_batch support enable_dp_attention (#4058) fzyzcjy 2025-04-09 14:44:25 +08:00
• 76c48a0913 [DeepEP] fix: import buffer error (#5179) Cheng Wan 2025-04-09 01:12:14 -04:00
• 90caf06c00 fix: use DeepEPDispatcher on CUDA (#5180) Yineng Zhang 2025-04-08 21:56:53 -07:00
• 6669d12707 feat: add DeepGEMM build warning (#5176) Yineng Zhang 2025-04-08 21:16:23 -07:00
• f2b70afde0 docs: remove the use of Downward API for LWS_WORKER_INDEX (#5110) Kay Yan 2025-04-09 11:46:11 +08:00
• bc3f6db2dd [Fix] DeepEP Compatibility with Low Latency (#5068) Jinyan Chen 2025-04-09 11:31:31 +08:00
• aac531c53b [Bugfix] Fix index out of bounds in local attention with large sequences (#5173) Chang Su 2025-04-08 18:43:13 -07:00
• 39efad4fbc Tiny disable model that does not work (#5175) fzyzcjy 2025-04-09 09:42:37 +08:00
• 466899e69c Fix multimodal hashing error (#5174) fzyzcjy 2025-04-09 09:42:26 +08:00
• 11d760d56a FP4 weight loading and inference (2/2) (#3972) Trevor Morris 2025-04-08 17:26:21 -07:00
• 5039d54772 Support 2x8xH100 for Llama 4 (#5159) fzyzcjy 2025-04-09 05:55:14 +08:00
• d09a51f1f6 [feat&refactor] Enhance multimodal input support with refactor io_struct (#4938) XinyuanTong 2025-04-08 14:48:07 -07:00
• f8194b267c Small improvement of native api docs (#5139) simveit 2025-04-08 21:09:26 +02:00
• 6d3b35fae9 [PD] Simplify mini LB (#4911) Byron Hsu 2025-04-08 09:42:34 -07:00
• a73c4df438 Add optimized native kernels in sgl-kernel (#5150) Ma Mingfei 2025-04-09 00:37:46 +08:00
• 89a554181f [PD] Fix unclosed prefill connection warning of mini_lb (#5155) shangmingc 2025-04-09 00:15:06 +08:00
• 2695ab0537 Fix loading KV quantization scale; Enable modelopt kv cache (#4686) Yun Dai 2025-04-08 09:11:35 -07:00
• 88d6fd9a11 Fix torch compile errors (#5158) kk 2025-04-08 23:04:37 +08:00
• cc88d98ab8 fix empty_cache error in pt_weights_iterator (#5151) DangKai 2025-04-08 16:22:10 +08:00
• 3033c11a21 Add dummy grok test to amd CI. (#5115) saienduri 2025-04-08 00:44:59 -07:00
• fd5a55cfd3 Use public model for FA3 speculative decode testing (#5152) Yubo Wang 2025-04-08 00:08:25 -07:00
• 804d9f2e4c Add unit test on page_size > 1 and mla and integration test for Flash Attention 3 (#4760) Yubo Wang 2025-04-07 23:20:51 -07:00
• a7c3f74bec [FA3 Feature] Support multi modal Llama-3.2-11B-Vision-Instruct (#5103) Chunan Zeng 2025-04-07 22:58:08 -07:00
• 5a144a8ab9 Fix run time error in ROCm platform (#5147) kk 2025-04-08 13:49:40 +08:00
• 27f8e6b9c1 fix multimodal hash feature (#5083) huangtingwei 2025-04-08 13:43:23 +08:00
• afb752bcbe [AMD] Fix missing per_token_group_quant_fp8 for ROCm (#5140) Hubert Lu 2025-04-07 22:38:25 -07:00
• 9731eca77b [modelopt] automatically inspect if model is ModelOpt quantized and set quantization method (#5145) Yun Dai 2025-04-07 22:12:11 -07:00
• 7c5658c189 feat: disable grammar restrictions within reasoning sections (#4984) mlmz 2025-04-08 12:46:47 +08:00
• 9798e72baa [Misc] Use pytest.mark.skipif in sgl-kernel test (#5137) yinfan98 2025-04-08 12:35:14 +08:00
• ade714a67f Add Llama4 user guide (#5133) Ke Bao 2025-04-08 10:09:34 +08:00
• 93470a1411 Refactor and Optimize FA3 Code (#5090) Stefan He 2025-04-07 11:52:42 -07:00
• db452760e5 [ci] fix llama4 ci error (#5126) Xiaoyu Zhang 2025-04-07 21:15:46 +08:00
• 57f99608f4 bump v0.4.5 (#5117) Yineng Zhang 2025-04-07 00:35:00 -07:00
• 819924748a Fix refactor error - fp8.py (#5106) HAI 2025-04-07 00:34:08 -07:00
• f04c80dc42 Add Llama4 support (#5092) Chang Su 2025-04-07 00:29:36 -07:00
• d1bb171180 Fix: Reduce the number of document ci attempts to avoid long ci running (#5097) mlmz 2025-04-06 15:43:48 +08:00
• 35e0856b90 bump v0.4.4.post4 (#5091) Yineng Zhang 2025-04-05 15:36:17 -07:00
• aba5ca154d python transfer custom allreduce from trt kernel to vllm kernel (#5080) Yi Zhang 2025-04-06 06:35:55 +08:00
• 496dde8491 bump sgl-kernel 0.0.8 (#5089) Yineng Zhang 2025-04-05 14:28:04 -07:00
• bcbbf519f9 sgl-kernel transfer custom allreduce from trt kernel to vllm kernel (#5079) Yi Zhang 2025-04-06 05:23:20 +08:00
• 0d99adb715 upgrade transformers 4.51.0 (#5088) Yineng Zhang 2025-04-05 14:20:23 -07:00
• efbae697b3 [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) Baizhou Zhang 2025-04-05 01:23:02 -07:00
• ca8d02abd5 FA3 Spec Decoding to support top k = 1 and add cuda graph support (#5050) Stefan He 2025-04-04 23:03:59 -07:00
• 3f287b8579 support sgl-kernel on blackwell (#5074) Yineng Zhang 2025-04-04 16:59:32 -07:00
• 7ed77d6b9e fix dummy-load deepseekv2 (#4535) inkcherry 2025-04-05 06:22:37 +08:00
• 4c54f44202 [deepep] fix: shared experts are not initialized when shared experts fusion is enabled (#5072) Cheng Wan 2025-04-04 18:08:30 -04:00
• 924ca7c92c Add DeepSeek V3/R1 shared experts fusion (#4918) Xiaoyu Zhang 2025-04-04 16:59:29 +08:00
• 6ff9c6a5e7 Cleanup unused resources after DeepEP operation (#4996) fzyzcjy 2025-04-04 15:36:04 +08:00
• 77e929a1a2 Support async DeepEP by splitting into two stages (#4995) fzyzcjy 2025-04-04 15:32:27 +08:00
• febe21ce03 Small refactor DeepEPDispatcher into subclasses (#4994) fzyzcjy 2025-04-04 15:24:18 +08:00
• a995a773a0 [fix] remove cuda_device_count_stateless (#5060) JieXin Liang 2025-04-04 15:18:26 +08:00
• 31035dda44 Add H20 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5057) Tommy Yang 2025-04-04 13:14:28 +08:00
• 913e38dffa Feature/revise docs ci (#5056) renxin 2025-04-03 21:20:21 -07:00
• d95269f9b3 [2/3] fix dsv3 awq issue (#4625) AniZpZ 2025-04-04 08:36:39 +08:00
• e53bf190bc upgrade sgl-kernel v0.0.7 (#5049) Yineng Zhang 2025-04-03 17:07:54 -07:00
• 3289c1207d Update the retry count (#5051) Yineng Zhang 2025-04-03 17:07:38 -07:00
• 69df9761dd Add LlavaLlamaForCausaLM in MultiModal Processors (#5039) Ravi Theja 2025-04-04 04:11:12 +05:30
• 98f768d194 update eagle-3 docs (#4796) simveit 2025-04-04 00:24:41 +02:00
• d7954b7682 bump sgl-kernel v0.0.7 (#5046) Yineng Zhang 2025-04-03 13:38:13 -07:00
• 74885a848b Revert "Replace enable_flashinfer_mla argument with attention_backend" (#5048) Lianmin Zheng 2025-04-03 13:30:56 -07:00
• b8b6008f47 [Fix] fix fa3 build at cu118 (#5036) yinfan98 2025-04-04 02:52:35 +08:00
• 8e10fec9a8 Small refactor DeepEPMode to clean up code a bit (#4992) fzyzcjy 2025-04-03 17:56:44 +08:00
• e8999b13b7 Replace enable_flashinfer_mla argument with attention_backend (#5005) Baizhou Zhang 2025-04-03 02:53:58 -07:00
• 772d2a191d try to fix ci oserror (#5024) Xiaoyu Zhang 2025-04-03 17:45:05 +08:00
• 9d0b36c47a fix deepgemm as well (#5030) Zhiqiang Xie 2025-04-03 02:41:37 -07:00
• 7d8c0ce7ce [Build] Support build sgl-kernel with ccache (#5020) Yuhong Guo 2025-04-03 15:22:37 +08:00
• e41549c3d6 fix: fix illegal cuda memory access at fused_moe_kernel (#4727) saltyfish66 2025-04-03 15:07:32 +08:00
• cccfc10e9c Feature/revise docs ci (#5009) renxin 2025-04-02 20:08:56 -07:00
• a2aea59b6e update cutlass tag (#5011) Zhiqiang Xie 2025-04-02 18:30:30 -07:00
• 2c8fd99363 [sgl-kernel] per token group quant support COLUMN MAJOR (#4817) Xiaoyu Zhang 2025-04-03 09:29:59 +08:00
• 31da75abed Update tokenizer_manager.py (#5008) Kaiyu Yang 2025-04-02 16:56:19 -04:00
• e983e43248 Add Eagle Speculative Decoding to FA3 Backend (#4951) Qingquan Song 2025-04-02 13:09:02 -07:00
• e9c6ce461d sgl scaled_fp8_quant support output padding (#4861) Xiaoyu Zhang 2025-04-02 23:53:57 +08:00
• 3fadc64793 bug fix for hicache host eviction (#4989) Zhiqiang Xie 2025-04-02 00:33:50 -07:00
• e119f04215 Large page size aligned hierarchical caching (#4581) Zhiqiang Xie 2025-04-01 22:38:15 -07:00
• 9eb49e878b [VLM RLHF] Take Image input for verl vlm rollout (#4915) XinyuanTong 2025-04-01 20:03:17 -07:00
• 12047f5e94 Prevent memory leak of retract_decode when page_size > 1 (#4977) Zhiqiang Xie 2025-04-01 15:30:45 -07:00
• fda6bb78da update bench_serving (#4958) Yineng Zhang 2025-04-01 15:10:56 -07:00