Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• bf863e3bbf fix: use sgl-kernel 0.3.5 (#9565) Yineng Zhang 2025-08-24 15:46:47 -07:00
• 9e169ea8b5 [router] add right rustls dependency in sgl-router cargo.toml (#9498) Bruce-x-1997 2025-08-25 00:03:15 +08:00
• e0ab167db0 chore: bump v0.5.1.post1 (#9558) Yineng Zhang 2025-08-24 01:14:17 -07:00
• c807cd7c75 chore: update configurer (#9557) Yineng Zhang 2025-08-24 01:05:00 -07:00
• 327f7b7c87 fix(grok): remove duplicate replicate_lm_head configuration (#9549) Vincent Zhong 2025-08-23 22:49:24 -04:00
• 80425e59bb [doc] deepseekv31 support (#9544) Xiaotong Jiang 2025-08-23 16:54:58 -07:00
• af9d4eb038 [readme] Include additional resources for the SGLang x AMD SF Meetup event (#9547) Mingyi 2025-08-23 16:51:16 -07:00
• fb107cfd75 feat: allow use local branch to build image (#9546) gongwei-130 2025-08-23 16:38:30 -07:00
• 97a38ee85b Release 0.5.1 (#9533) Lianmin Zheng 2025-08-23 07:09:26 -07:00
• 86d10d220f Update grok.py and tiktoken tokenizer (#9532) Lianmin Zheng 2025-08-23 05:40:18 -07:00
• 83871aa12d feat(hicache): Supports 3fs-hicache compatibility with dp-attention (#9372) hzh0425 2025-08-23 17:08:32 +08:00
• b1b3f0b38f Partially unify triton per token group quant kernels (#9485) fzyzcjy 2025-08-23 17:07:31 +08:00
• 34e5e11f0f Tiny make device_loading_context more static (#9478) fzyzcjy 2025-08-23 17:07:15 +08:00
• 2600fc0d47 Overlapped weight offload (#8034) fzyzcjy 2025-08-23 17:06:46 +08:00
• ccd3fb946e [fix] Fix mxfp4 triton MoE tp bug (#9473) hlu1 2025-08-23 01:48:40 -07:00
• c9dd70fbde tool-call(dsv3): Improve deepseek-v3 chat template and tool_choice = required (#9525) Chang Su 2025-08-23 01:46:56 -07:00
• 6b2b8bf0e1 fix: blackwell dsv3 fp8 issue temporary solution (#9530) Yineng Zhang 2025-08-23 01:33:21 -07:00
• 4edbe0d534 [benchmark] Add benchmark scripts for ceval and boolq (#8946) yuxingcyx 2025-08-23 15:40:15 +08:00
• 0374304a2c Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004) fzyzcjy 2025-08-23 15:38:40 +08:00
• 127d4b0d5e Support GC Freezing to improve latency & throughput (#9241) Chanh Nguyen 2025-08-22 22:43:09 -07:00
• 7e880286b5 Add support for extensions of interface and pre-registrations to NIXL HiCache (#9211) Moein Khazraee 2025-08-22 20:06:13 -07:00
• 446c8e4cdb [router] ignore client error when record failure in pd_router (#9503) Bruce-x-1997 2025-08-23 05:19:45 +08:00
• 5ef545e678 [router] Move all protocols to spec.rs file (#9519) Keyang Ru 2025-08-22 14:18:47 -07:00
• c4500233ff Add Qwen3-30B-A3B-Thinking-2507 support on AMD GPUs. (#9456) sogalin 2025-08-22 13:14:42 -07:00
• f445a1d9a3 [AMD] Fix Llama 4 FP8 accuracy issues on MI300X (#7699) Hubert Lu 2025-08-22 13:13:45 -07:00
• e5638573c1 [NVIDA] [1/N] Nvfp4 Masked Gemm: Add quant op for the flashinfer grouped gemm (#9200) Kaixi Hou 2025-08-22 12:19:45 -07:00
• f556ac8bd8 [router] add json tool parser (#9516) Simo Lin 2025-08-22 12:13:04 -07:00
• 110a65989b [MTP] Force greedy sampling on AMD (#9127) datdo-msft 2025-08-22 11:14:43 -07:00
• 49f9d02538 [router] tokenizer arch doc (#9513) Simo Lin 2025-08-22 09:52:33 -07:00
• 0f587e80d3 Use Tensor Core Decode when gqa group size >= 4 (#8624) Wenxuan Tan 2025-08-22 10:25:15 -05:00
• 6078d5fcc0 [HiCacheStorage] backup optimization for MLA model (#8865) huangtingwei 2025-08-22 18:03:51 +08:00
• 70cf4abccc 3fs zerocopy (#9109) pansicheng 2025-08-22 17:56:38 +08:00
• cebf45994b [bugfix] Make --enable-hierarchical-cache and --disable-radix-cache mutually exclusive (#9452) Xuchun Shang 2025-08-22 17:49:52 +08:00
• 9c0c1e30b2 Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507) Qiaolin Yu 2025-08-22 02:05:02 -07:00
• a1f011d09a minor: determine mm attn backend based on platforms (#9303) Mick 2025-08-22 16:08:41 +08:00
• 9ec314c6ac Support speculative decoding in the trtllm_mha attention backend (#9331) Qiaolin Yu 2025-08-21 23:53:35 -07:00
• fedfe91c1a [Docs] Add doc and quick demo for gpt-oss responses api & buildin tools (#9497) Xinyuan Tong 2025-08-22 14:51:52 +08:00
• 988accbc1e Update docker file for supporting PD-Disaggregation on MI300x (#9494) kk 2025-08-22 14:48:40 +08:00
• b6b2287e4b chore: bump sgl-kernel v0.3.6.post2 (#9475) Yineng Zhang 2025-08-21 23:02:08 -07:00
• 243e745d07 Add trtllm_mla and cutlass_mla for ragged fmha for chunked prefill (#9480) Elfie Guo 2025-08-21 23:01:36 -07:00
• 61a0e600df torch.compile() mrope (#9487) timmy-feng 2025-08-22 02:01:08 -04:00
• 0f8cee8cd3 [router] fix router load guard tracking for streaming (#9491) Simo Lin 2025-08-21 22:48:29 -07:00
• 816c4c8572 [router] add tool parser base structure and partial json parser (#9482) Chang Su 2025-08-21 22:08:56 -07:00
• 13ec8d427e [Docs]Update reasoning parser doc & fix outdated link (#9492) Xinyuan Tong 2025-08-22 13:08:28 +08:00
• 05bd789791 [docs]: fix reasoning context in docs (#9483) Chayenne 2025-08-21 20:04:12 -07:00
• 5fd311d33e [code clean] add H20 cutlass groupGemm default config (#9333) kousakawang 2025-08-22 10:23:29 +08:00
• 53e2cd464f [router] remove all tokenizer metrics for performance (#9474) Chang Su 2025-08-21 18:35:24 -07:00
• 9708d353b7 Support MHA with chunked prefix cache for flashinfer/flashmla backend, support page size > 1 for MHA chunked prefix (#8616) Yongfei Xu 2025-08-22 09:19:44 +08:00
• 704ced1b2e [AMD] Remove the deprecated C10_WARP_SIZE (#9356) Hubert Lu 2025-08-21 18:16:35 -07:00
• 3cc3d9b950 Add Support for Page Size greater than 1 for Flashinfer MLA Backend (#8593) Pavani Majety 2025-08-21 18:15:06 -07:00
• 0b3a5b1151 Update reasoning parser doc (#9468) Xinyuan Tong 2025-08-22 08:25:30 +08:00
• 6c855db82c Revert "bugfix: Fix output_ids extraction in detokenizer_manager" (#9467) Xinyuan Tong 2025-08-22 08:24:25 +08:00
• 0f9318f7d0 feat: update auto_choose_speculative_params (#9470) Yineng Zhang 2025-08-21 17:12:12 -07:00
• 849957bc76 fix: tmp revert gpt oss tp sharding on hopper (#9469) Yineng Zhang 2025-08-21 17:03:21 -07:00
• cded039b57 [FA3] Init Spec Page Table only when Spec is enabled to save ~40MB (#9455) Stefan He 2025-08-21 15:11:38 -07:00
• 275f9df381 feat: add fused moe config for GLM-4.5-Air-FP8 on B200 (#9463) zixuanzhang226 2025-08-21 15:10:20 -07:00
• e8449ab515 Add deepseek v3.1 thinking parser support and update docs (#9464) Xinyuan Tong 2025-08-22 06:09:40 +08:00
• 4746aaea41 fix: support fb fp8 (#9462) Yineng Zhang 2025-08-21 14:31:43 -07:00
• 10d34f74e2 fix: should return a invalid request response when schema missing (#9461) gongwei-130 2025-08-21 14:06:50 -07:00
• 9ba7253094 accomendate reasoning_effort set in chat_template_kwargs (#9458) gongwei-130 2025-08-21 13:22:03 -07:00
• 9c8e4f69c3 [5/n]decouple quantization implementation from vLLM dependency (#9454) Hongbo Xu 2025-08-22 03:52:07 +08:00
• 78ae175866 [router] add tokenizer benchmark (#9427) Simo Lin 2025-08-21 11:09:39 -07:00
• dae9a80f43 [fix] Fix mxfp4 weight loading bug with TP sharding in GPT-OSS (#9433) hlu1 2025-08-21 03:50:51 -07:00
• e85cb1ce9d Fix quant kernel test errors and benchmark wrong output speeds (#7604) fzyzcjy 2025-08-21 18:48:41 +08:00
• 55d336cb08 Refactor weight offloading logic (#8521) fzyzcjy 2025-08-21 18:48:13 +08:00
• de4990a5b2 [Bug] Fix w4afp8 moe kernel (#9392) Yuhao Yao 2025-08-21 18:45:18 +08:00
• 029e0af31d ci: enhance xeon ci (#9395) DiweiSun 2025-08-21 18:35:17 +08:00
• 64574ef8c0 Enables speculative decoding for the trtllm_mla attention backend (#9238) pranavm-nvidia 2025-08-21 01:18:21 -07:00
• 18da2c96ec [NVIDIA] Fix trtllm fp4 moe backend when used in MTP (#9384) Kaixi Hou 2025-08-21 00:54:01 -07:00
• 9b5f0f64f5 Fix tiny misalign with previous truncation setting in tokenizer_manager (#9430) Liangsheng Yin 2025-08-21 14:05:35 +08:00
• 70bb066ee4 Fix FP4 inference corruption issue in glm4.5-air model (#9346) Azure 2025-08-21 13:13:47 +08:00
• 2c4b4b786b [feature] Ascend NPU graph support (#9399) VDV1985 2025-08-21 07:13:27 +03:00
• 7cd2ee06d7 feat: Add Triton fallback option and SM120 MoE configs for FP8 models (#9251) Martin Vit 2025-08-21 04:33:15 +02:00
• eb19ccadae [bug] fix errors related to context length in SD (#9388) Liangsheng Yin 2025-08-21 10:32:34 +08:00
• 25ef53f05f [PD] Fix nvlink transport accuracy through transferring metadata with tcp (#9261) Shangming Cai 2025-08-21 10:29:10 +08:00
• c674bf9c6b Fix biased_grouped_topk_cpu (#9420) Cao E 2025-08-21 10:18:48 +08:00
• af1973b871 Fix max_seq_len_k in trtllm_mha attention backend (#9416) Qiaolin Yu 2025-08-20 19:17:13 -07:00
• 5cfbb4c136 [router] add glm and step3 reasoning parser (#9415) Chang Su 2025-08-20 18:33:10 -07:00
• e65231022f [router] add tokenizer integration test with real mini tokenizer (#9413) Chang Su 2025-08-20 17:56:23 -07:00
• 3828db4309 [router] Add IGW (Inference Gateway) Feature Flag (#9371) Keyang Ru 2025-08-20 17:38:57 -07:00
• 88fbc31b50 Support trtllm_allreduce_fusion in flashinfer for cuda<12.8 (#9339) strgrb 2025-08-21 07:54:30 +08:00
• 8f5b9910c1 Add support for Qwen3-seq-cls (#9357) nathan 2025-08-21 01:51:56 +02:00
• ef3004d90a misc: parse bench_serving result as markdown table (#9377) Mick 2025-08-21 07:44:20 +08:00
• 84719b527a fix: InternS1 don't recognize image, updates image token for InternVL processor (#9381) Xinyuan Tong 2025-08-21 07:43:03 +08:00
• e99729c9f3 Fixed the issue where eagle3 TPOT was not as good as without eagle3. (#9404) jiapingW 2025-08-21 07:42:01 +08:00
• c10b8e6a0f Support DP attention with GPT-OSS (#9359) Nicolas Castet 2025-08-20 18:36:31 -05:00
• d4bce29721 Fix incorrect logic in chat template handling. (#9336) Lifu Huang 2025-08-20 16:25:36 -07:00
• b0980af89f Support pinning adapter via server args. (#9249) Lifu Huang 2025-08-20 16:25:01 -07:00
• 24eaebeb4b Fix FlashInfer GPU <-> CPU sync (#9409) Nathan Wang 2025-08-20 18:26:12 -04:00
• a91e90d9a3 [2/2] Fuse routed scaling factor into select_experts (#8690) Trevor Morris 2025-08-20 15:10:16 -07:00
• f96413c444 Refactor allreduce add rmsnorm pattern (#9278) Xiaoyu Zhang 2025-08-20 17:03:08 +08:00
• 08ebdf79d0 Fix the --allow-auto-truncate argument in tokenizer manager. (#9391) Liangsheng Yin 2025-08-20 16:56:47 +08:00
• 42c8704560 Add PDL support for quant kernel and rope kernel (#9106) fzyzcjy 2025-08-20 16:56:29 +08:00
• c9bf3877a0 Reduce overhead for fa by not calling heavy CUDA property check (#7375) Yichen Yan 2025-08-20 16:26:28 +08:00
• de2dd73831 Revert "[feature] Rework Ascend NPU graph support" (#9385) Even Zhou 2025-08-20 15:35:10 +08:00
• 1ec9769753 [Docs] Update contribution guide (#9383) Lianmin Zheng 2025-08-19 23:37:45 -07:00
• d8ed60f254 [CI] Fix disaggregation failure tolerance CI (#9378) Shangming Cai 2025-08-20 14:31:08 +08:00
• f1b0eda55c [readme] Add SGLang x AMD SF meetup information (#9380) Mingyi 2025-08-19 22:25:09 -07:00
• f20b6a3f2b [minor] Sync style changes (#9376) Lianmin Zheng 2025-08-19 21:35:01 -07:00
• 3680d6f88b [feature] Rework Ascend NPU graph support (#9350) Even Zhou 2025-08-20 11:32:27 +08:00