Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 7c1692aa90 Docs: Reorngaize dpsk links (#3900) Chayenne 2025-02-26 15:16:31 -08:00
• 8f019c7d1a Docs: Move dpsk docs forward a step (#3894) Chayenne 2025-02-26 11:43:20 -08:00
• 7551498a69 [Feature] Support llguidance for constrained decoding (#3298) JC1DA 2025-02-26 10:41:49 -08:00
• 44a2c4bd56 Docs: improve link to docs (#3860) simveit 2025-02-26 19:29:25 +01:00
• c9fc4a9d26 Docs: delete sgl-kernel install in docs (#3845) simveit 2025-02-26 18:25:43 +01:00
• 21463e321a Expert Parallelism (EP) Support for DeepSeek V3/R1 (#3602) lukec 2025-02-26 18:29:37 +08:00
• 3dc9ff3ce8 [doc] fixed dpsk quant faq (#3865) Shenggui Li 2025-02-26 11:40:47 +08:00
• 06427dfab1 [doc] added quantization doc for dpsk (#3843) Shenggui Li 2025-02-26 01:43:28 +08:00
• 60524920ba [Bug]: Fix maximum recursion depth triggered on exception exit (#3519) Kebe 2025-02-26 01:39:38 +08:00
• 107710268a [BugFix] Fix crash when receive a req with structed output in DP attention mode. (#3841) IAN 2025-02-26 01:32:05 +08:00
• 4606e2a3fe Bug: fix capture_bs (#3857) who who who 2025-02-26 00:40:35 +08:00
• 127998cc41 Fix allgather ops inside cuda graphs (#3709) Nicolas Castet 2025-02-25 10:39:10 -06:00
• c0bb9eb3b3 [improve] made timeout configurable (#3803) Shenggui Li 2025-02-25 16:26:08 +08:00
• 7036d6fc67 [Bug]: Add missing clamp to llavavid (#3787) Yueyang Pan 2025-02-25 04:10:15 +01:00
• 6ce9dbe828 [ROCm] Enable Fused MLA Triton kernel for DeepSeekV3 (#3237) Chaitanya Sri Krishna Lolla 2025-02-25 07:44:31 +05:30
• 3758d209a0 [Doc] Fix typo in server-argument description (#3641) Yuanheng Zhao 2025-02-25 08:57:13 +08:00
• faf29e0b23 Docs: fix doc site copyright to current year (#3741) Wilson Wu 2025-02-25 08:56:04 +08:00
• b0743ea059 Docs: fix dead link in router.md (#3799) He1pa 2025-02-25 08:53:57 +08:00
• 60b771c815 Improve: fix typos (#3801) Wang Ran (汪然) 2025-02-25 08:51:23 +08:00
• d7934cde45 Fix CI and install docs (#3821) Lianmin Zheng 2025-02-24 16:17:38 -08:00
• 62bbd34393 Revert "Extract generation_manager from tokenizer_manager" (#3829) Lianmin Zheng 2025-02-24 14:49:16 -08:00
• f2388f6b95 Revert "Rename TokenizerManager to StdOrchestrator" (#3828) Lianmin Zheng 2025-02-24 14:47:59 -08:00
• c9745ee082 Fix pandas dependency in CI (#3818) Lianmin Zheng 2025-02-24 05:56:57 -08:00
• 1a6e97577a Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) laixin 2025-02-24 21:43:35 +08:00
• b110084654 Refactor flashinfer logic for deepseek v3 and fix accuracy bug (#3785) Baizhou Zhang 2025-02-24 04:07:25 -08:00
• 27a46317b6 Fix dependency (#3813) Lianmin Zheng 2025-02-24 03:50:58 -08:00
• c979580817 Update readme (#3809) Lianmin Zheng 2025-02-24 00:31:08 -08:00
• 6c7a152c5a Hierarchical Caching for SGLang (#2693) Zhiqiang Xie 2025-02-23 21:56:30 -08:00
• 4d2a88bdff [Docs]Add instruction for manually stopping nsys profiler (#3795) Baizhou Zhang 2025-02-23 13:21:48 -08:00
• 45360b2fa9 Improve: Rename TokenizerManager to StdOrchestrator (#3116) fzyzcjy 2025-02-23 16:30:58 +08:00
• 3f41b18455 Improve: Extract generation_manager from tokenizer_manager (#3115) fzyzcjy 2025-02-23 15:25:45 +08:00
• 45205d88a0 bench: Add MMMU benchmark for vLM (#3562) Mick 2025-02-23 00:10:59 +08:00
• 9087694006 Improve: Use TypeBasedDispatcher in DetokenizerManager (#3117) fzyzcjy 2025-02-22 11:50:46 +08:00
• a3339d8cac Bug: Fix weight loader error when LM head weights are tied (#3766) fzyzcjy 2025-02-22 09:53:12 +08:00
• 14d90617b0 Bug: fix lm head weights in Qwen models (#3777) Chayenne 2025-02-21 16:49:31 -08:00
• d37f95511d Improve: Tiny fix Olmo2 (#3348) fzyzcjy 2025-02-22 08:09:35 +08:00
• c66b2c9cf1 Add support for nvidia modelopt fp8 kv cache (#3223) Zhiyu 2025-02-21 15:04:58 -08:00
• 20b765a26e Model: Support Qwen 72B RM model. (#3772) simveit 2025-02-21 23:38:21 +01:00
• e310722266 Docs: Update offline_engine_api and add links (#3773) Chayenne 2025-02-21 14:15:52 -08:00
• e074e76b31 docs: Add offline engine launch example and documentation (#3771) Shi Shuai 2025-02-21 19:25:52 +00:00
• 4592afc27d Docs: Fix layout to docs (#3733) simveit 2025-02-21 20:24:13 +01:00
• 9af0e21ef5 [bug] fixed batch api for DeepSeek V3/R1 (#3754) Shenggui Li 2025-02-22 02:28:16 +08:00
• c7c79b16cd [Fix] OpenAI API adapter tokenizer encoding (#3432) Shi Shuai 2025-02-21 17:24:15 +00:00
• d8d75d256a Change description of nvidia jetson docs (#3761) Shakhizat Nurgaliyev 2025-02-21 17:44:22 +05:00
• 1df6eabd5d feat: Add SageMaker support (#3740) Andrew Smith 2025-02-21 22:31:09 +11:00
• 0c227ee373 feat: update grouped_topk to support softmax and sigmoid (#3680) zixuanzhang226 2025-02-21 00:30:15 -08:00
• 5c54ef0352 AMD/ROCm: update AITER repo to ROCm/aiter (#3747) HAI 2025-02-21 00:18:08 -08:00
• c6a4852136 [docs] added torch.compile cache to dpsk manual (#3737) Shenggui Li 2025-02-21 16:11:40 +08:00
• 4f678c87d8 docs: fix 404 link (#3588) Trayvon Pan 2025-02-21 07:15:57 +08:00
• e79f7420be [Fix] Fix bugs and refactor codes in lora for better scalability. (#3652) aoshen524 2025-02-20 14:51:57 -05:00
• ac05310098 [Docs] Modify ep related server args and remove cublas part of deepseek (#3732) Baizhou Zhang 2025-02-20 11:37:56 -08:00
• d5d80ab477 [Bugfix] Fix scores mask for moe topk (#3705) chenxiaobing 2025-02-21 02:17:23 +08:00
• ddcf9fe3be Optimize triton attention custom mask (#3731) Ke Bao 2025-02-21 00:54:41 +08:00
• 6252ade985 revert BLOCK and num_warps on HIP (#3722) HAI 2025-02-20 07:30:18 -08:00
• 1eb8eade2b add control for cutlass fp8 blockwise gemm (#3727) yizhang2077 2025-02-20 16:10:35 +08:00
• 3c7bfd7eab Docs: Fix layout with sub-section (#3710) Chayenne 2025-02-19 15:44:30 -08:00
• bb121214c2 Variance measure for reasoning benchmark (#3677) simveit 2025-02-19 20:49:49 +01:00
• 55de40f782 [Docs]: Fix Multi-User Port Allocation Conflicts (#3601) Shi Shuai 2025-02-19 19:15:44 +00:00
• 6b0aeb58fd [moe] optim: reduce memory consumption in fused_moe (#3692) Cheng Wan 2025-02-19 13:25:05 -05:00
• bb3e526823 [k8s] remove unnecessary hostIPC for security concern (#3700) Peter Pan 2025-02-20 02:11:21 +08:00
• f93e915817 [Docs] Add SkyPilot DeepSeek example (#3706) Zhanghao Wu 2025-02-19 10:10:23 -08:00
• 55a7ec388f use warp shuffle style reduce and flashinfer vectorize (#3628) Xiaoyu Zhang 2025-02-19 20:53:51 +08:00
• fe0673f1cc set NCCL_IB_GID_INDEX=3 for multi node NVIDIA InfiniBand if needed (#3698) Yineng Zhang 2025-02-19 20:50:22 +08:00
• 99c1b9d2ee fix: apply cache size limit of attention mask for VisionAttention (#3657) Mick 2025-02-19 20:16:48 +08:00
• 634a3561ac AMD Prefill optimize (#3665) who who who 2025-02-19 01:35:58 +08:00
• 424848d26f fix: remove dependency on latest transformers impl (#3635) Mick 2025-02-19 01:14:11 +08:00
• e5ce395a6c Fix draft decode max batch size (#3676) Ke Bao 2025-02-18 23:03:26 +08:00
• f983213a1f update pr-test (#3663) Yineng Zhang 2025-02-18 17:23:43 +08:00
• 67fc595bb8 [Feature] Apply Cublas Grouped Gemm kernel (#3629) Baizhou Zhang 2025-02-17 23:18:31 -08:00
• 07ab4d4a2d fix #3654 Yineng Zhang 2025-02-18 15:16:16 +08:00
• 522e18eaeb Update amd docker image. (#3654) saienduri 2025-02-17 20:12:55 -08:00
• c51dc2cc8d Docs: Deploy multi-node inference (LWS method) using sglang in a K8s cluster (#3624) ybyang 2025-02-18 10:14:20 +08:00
• ddf39d3fce [ROCm] Optimal MOE Tuning for AMD Radeon Graphics (#3567) yigex 2025-02-18 09:54:10 +08:00
• 2eab113206 [ROCm] Add additional block quant GEMM tuning configs for AMD GPUs. (#3616) Wen-Heng (Jack) Chung 2025-02-17 17:54:18 -06:00
• 058d199d4e use transformers 4.48.3 (#3650) Yineng Zhang 2025-02-18 04:40:47 +08:00
• a5375adc3a chore: bump v0.4.3.post2 (#3645) Yineng Zhang 2025-02-18 02:48:30 +08:00
• 75d171a9c5 chore: update flashinfer v0.2.1.post2 (#3644) Yineng Zhang 2025-02-18 02:47:42 +08:00
• 714f3e6362 feat: support flashinfer mla with prefix cache (#3643) Yineng Zhang 2025-02-18 02:06:43 +08:00
• c38f3aed24 support multi-gpu block-gemm tuning (#3639) Xiaoyu Zhang 2025-02-18 00:00:35 +08:00
• 2e6be53e7d fix Dockerfile.rocm Yineng Zhang 2025-02-17 22:13:03 +08:00
• e782eb7e6a chore: bump v0.4.3.post1 (#3638) Yineng Zhang 2025-02-17 21:58:19 +08:00
• e319153be8 update unit test (#3636) Yineng Zhang 2025-02-17 21:06:10 +08:00
• 32b44d2fca add mtp unit test (#3634) Yineng Zhang 2025-02-17 19:04:07 +08:00
• 5f1a485d9e Revert "[ROCm] Use tl.range() in block GEMM kernels with `num_stage… (#3632) Yineng Zhang 2025-02-17 18:01:21 +08:00
• c9565e49e7 [docker] added rdma support (#3619) Shenggui Li 2025-02-17 15:36:16 +08:00
• d03c4c25a7 [docs] Update sampling_params.md (#3617) Shi Shuai 2025-02-17 02:52:30 +00:00
• 8f13377dea Draft of updated doc for sampling params. (#3260) simveit 2025-02-16 23:28:22 +01:00
• 3d4a8f9bc0 Benchmark for reasoning models (#3532) simveit 2025-02-16 20:07:30 +01:00
• 7474bed883 Update to latest amd image. (#3597) saienduri 2025-02-16 08:29:47 -08:00
• 03caefeb51 [ROCm] Use tl.range() in block GEMM kernels with num_stages set by host. (#3535) Wen-Heng (Jack) Chung 2025-02-16 03:40:38 -06:00
• bcc213df61 Model: Support Qwen 2.5 vl (#3258) Mick 2025-02-16 16:58:53 +08:00
• 39416e394a fix lockfile and port_registry file permission error (#3598) Jiada Li 2025-02-15 22:14:45 -05:00
• 231c40d859 [docs] added favicon to sphinx html (#3564) Shenggui Li 2025-02-16 02:21:21 +08:00
• bbc47c348f fix apply_token_bitmask_inplace_cuda (#3594) Yineng Zhang 2025-02-15 23:55:08 +08:00
• dfce926921 fix high qps crash when enable mtp (#3592) Yineng Zhang 2025-02-15 23:11:28 +08:00
• 6718b10996 fix eagle unit test (#3591) Yineng Zhang 2025-02-15 23:10:48 +08:00
• 7711ac6ed0 doc: emphasize and notify the usage of chat_template (#3589) Mick 2025-02-15 16:10:32 +08:00
• 7443197a63 [CI] Improve Docs CI Efficiency (#3587) Shi Shuai 2025-02-15 03:57:00 +00:00
• 862dd76c76 Support NextN (MTP) speculative decoding for DeepSeek-V3/R1 (#3582) Ke Bao 2025-02-15 05:28:34 +08:00
• fb4c9c3a30 [fix] added support for vlm in offline inference (#3548) Shenggui Li 2025-02-15 05:27:29 +08:00