Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• fb915bd1a2 Disable overlap scheduler for multimodal models (#2235) Lianmin Zheng 2024-11-27 23:44:33 -08:00
• 09798b36cd Fix chunked prefill size for bench_offline_throughput (#2234) Lianmin Zheng 2024-11-27 23:37:20 -08:00
• b79fffdcb5 Update Install Method 2. From source (#2232) HAI 2024-11-27 22:46:55 -08:00
• cd51758fad Rename tuned MI300X config files for fused_moe_triton (#2228) HAI 2024-11-27 21:18:51 -08:00
• 91e5dbf554 add profile in offline benchmark & update doc (#2123) bjmsong 2024-11-28 06:57:13 +08:00
• dd5eba4c88 Remove fused_moe_grok (#2223) Lianmin Zheng 2024-11-27 14:28:55 -08:00
• a4fd2f9b46 fix typo prompts (#2224) Baoyuan Qi 2024-11-28 04:07:00 +08:00
• 92d1253e58 Bump sglang-router to 0.0.10 for env name change (#2226) Byron Hsu 2024-11-27 11:23:32 -08:00
• a9ca297d76 [3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs (#2191) kk 2024-11-28 02:23:10 +08:00
• 2a02185c5f Rename DP_RANK to SGLANG_DP_RANK (#2218) Lianmin Zheng 2024-11-27 09:36:36 -08:00
• fed4c6946a Release v0.3.6.post2 (#2214) Lianmin Zheng 2024-11-27 03:35:30 -08:00
• fb6e04a0c2 Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2222) Lianmin Zheng 2024-11-27 02:52:46 -08:00
• 6997e28f6e Revert "Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default" (#2221) Lianmin Zheng 2024-11-27 02:02:01 -08:00
• a0e58740a8 Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2217) Lianmin Zheng 2024-11-27 01:13:41 -08:00
• 37c8a5761f [feat] Support session control for vision language models (#2210) Ying Sheng 2024-11-27 00:03:29 -08:00
• c754652fcd Fix flasky tests (#2212) Lianmin Zheng 2024-11-26 23:06:20 -08:00
• 0b46b951ae Fix rust warning (#2208) Byron Hsu 2024-11-26 15:00:41 -08:00
• 2763c0a73a Bump router to 0.0.9 with better logging (#2207) Byron Hsu 2024-11-26 13:30:28 -08:00
• de3b67b77d docs: update adoption (#2204) Yineng Zhang 2024-11-27 04:57:16 +08:00
• 19f33b3237 add sglang version to get_server_info (#2206) Yudi Xue 2024-11-26 12:10:23 -08:00
• 30ce5b599e minor: update check_env (#2201) Yineng Zhang 2024-11-26 18:22:55 +08:00
• bc1f6fda0d fix: add cuda-python for xgrammar (#2199) Yineng Zhang 2024-11-26 17:24:18 +08:00
• 867e092f82 using is not not != to test None (#2196) Wang Ran (汪然) 2024-11-26 17:00:38 +08:00
• 88c7763f53 Remove unresolved reference 'self' (#2198) Andrew Lyu 2024-11-26 16:59:58 +08:00
• e4118b15b3 remove unused imports (#2195) Wang Ran (汪然) 2024-11-26 16:59:36 +08:00
• ba4ee37fa4 Update sampler.py to skip the success check (#2197) Lianmin Zheng 2024-11-26 00:58:57 -08:00
• ac5a0f0488 Release v0.3.6.post1 (#2189) Lianmin Zheng 2024-11-25 17:31:37 -08:00
• ea34350d88 Rename double sparsity config file (#2188) Lianmin Zheng 2024-11-25 17:12:08 -08:00
• 1605ae121e [CI] Minor fix for CI (#2187) Lianmin Zheng 2024-11-25 16:38:43 -08:00
• 1aea19f64b Input_embeds support (#2052) Rin Intachuen 2024-11-25 19:35:04 -05:00
• 1f76fc6e3f [router] Rust e2e test (#2184) Byron Hsu 2024-11-25 16:02:03 -08:00
• 7f076c2ce6 Update XGrammar to the latest API (#2176) Yixin Dong 2024-11-25 18:58:30 -05:00
• 3c5538f781 Update CI threshold (#2186) Lianmin Zheng 2024-11-25 15:24:17 -08:00
• 10189d08dd [Performance]: Process affinity to CPU cores with multiple sockets support (#2171) HAI 2024-11-25 14:57:32 -08:00
• c4336b2b60 Use custom allreduce w/ torch.compile (#2185) Lianmin Zheng 2024-11-25 14:55:01 -08:00
• 4d62bca542 [router] Replace print with logger (#2183) Byron Hsu 2024-11-25 13:36:02 -08:00
• e1e595d702 [feat] Refactor session control interface and add CI (#2173) Ying Sheng 2024-11-25 12:32:51 -08:00
• 5ada33ffa0 Bump rustls from 0.23.16 to 0.23.18 in /rust (#2182) dependabot[bot] 2024-11-26 03:22:33 +08:00
• 254fd130e2 [CI] Split test cases in CI for better load balancing (#2180) Lianmin Zheng 2024-11-25 04:58:16 -08:00
• 538fa0ae13 [Fix] Avoid calling fill_vocab_mask for terminated requests (#2175) Yixin Dong 2024-11-25 04:31:25 -05:00
• 55842eb81a feat: fused_moe fp8 monkey patch (#2174) Yineng Zhang 2024-11-25 17:06:36 +08:00
• a866b65e1d Bump rust router to 0.0.8 Byron Hsu 2024-11-24 23:17:38 -08:00
• 4b0a1c9365 Replace prob based with threshold based load balancing (#2170) Byron Hsu 2024-11-24 23:17:11 -08:00
• 8e1adb8441 Allow overwrite flashinfer use_tensorcore (#2169) Lianmin Zheng 2024-11-24 20:58:17 -08:00
• dd44173dad [Fused moe] add tuning fused configs for qwen2 57b and mixtral 8x7b (#2167) Xiaoyu Zhang 2024-11-25 10:37:50 +08:00
• 8912b7637f Fix docs (#2164) Lianmin Zheng 2024-11-24 08:25:56 -08:00
• be0124bda0 Rename triton_fused_moe -> fused_moe_triton (#2163) Lianmin Zheng 2024-11-24 08:12:35 -08:00
• fe5d3e818f Balance CI tests (#2162) Lianmin Zheng 2024-11-24 07:38:52 -08:00
• 731146f6cb Fix mixed chunked prefill in overlap mode (#2158) Lianmin Zheng 2024-11-24 07:17:37 -08:00
• fa27161380 fix: use torch.sum for compatible (#2161) Yineng Zhang 2024-11-24 22:37:04 +08:00
• 5652c56535 Update CI threshold & Improve code style (#2159) Lianmin Zheng 2024-11-24 06:29:38 -08:00
• e3938b2f9c feat: update other MoE models deps (#2156) Yineng Zhang 2024-11-24 21:36:34 +08:00
• c211e7b669 Simplify batch update (#2154) Lianmin Zheng 2024-11-24 04:47:10 -08:00
• d90c3d6b8b fix: resolve end-of-file-fixer (#2157) Yineng Zhang 2024-11-24 20:38:26 +08:00
• 9e8f8fbf95 feat: update gitignore and add tuning config for FusedMoE (#2155) Yineng Zhang 2024-11-24 20:24:58 +08:00
• b509db5832 feat: remove the dependency on FusedMoE (#2153) Yineng Zhang 2024-11-24 20:09:27 +08:00
• dbe1729395 Merged three native APIs into one: get_server_info (#2152) Henry Hyeonmok Ko 2024-11-24 01:37:58 -08:00
• 84a1698d67 Update release-pypi-router.yml Byron Hsu 2024-11-23 17:35:25 -08:00
• 32293a299c Improve sglang router (#2148) Byron Hsu 2024-11-23 17:34:24 -08:00
• 7921690893 add prefix match for certain tenant (#2147) Byron Hsu 2024-11-23 15:35:02 -08:00
• bbb81c2457 Add more api routes (completion, health, etc) to the router (#2146) Byron Hsu 2024-11-23 15:10:26 -08:00
• 52f58fc42a fix dp_rank env (#2144) Byron Hsu 2024-11-23 11:46:21 -08:00
• 145c0ddc2d update router doc (#2143) Byron Hsu 2024-11-23 11:01:04 -08:00
• 505d7f71a6 Bump sglang-router to 0.0.5 (#2142) Byron Hsu 2024-11-23 08:35:46 -08:00
• cbedd1db1d [router] cache-aware load-balancing router v1 (#2114) Byron Hsu 2024-11-23 08:34:48 -08:00
• ad47749b82 fix: resolve bench_serving args (#2139) Yineng Zhang 2024-11-23 17:45:42 +08:00
• 751c3a037c Fix dp print message (#2138) Lianmin Zheng 2024-11-23 01:22:26 -08:00
• 60769be14d Add concurrency option for benchmark (#2136) Yunmeng 2024-11-23 17:07:07 +08:00
• a78d8f8db3 [CI] Fix test cases (#2137) Lianmin Zheng 2024-11-23 01:00:07 -08:00
• c5f865013e Fix grid size in Triton decoding kernel (#2134) Ke Bao 2024-11-23 16:51:46 +08:00
• d98fa1e93d Add simple CPU offloading support. (#2081) Jani Monoses 2024-11-23 08:23:53 +02:00
• 865233e256 Add initial support for intel Gaudi accelerators (#2121) Ankur Neog 2024-11-23 09:52:23 +05:30
• 66d4859acf Revert "Only stream output on tp rank 0" (#2130) Lianmin Zheng 2024-11-22 15:46:16 -08:00
• e1b63624d7 Only stream output on tp rank 0 (#2124) Lianmin Zheng 2024-11-22 15:13:44 -08:00
• c35cd1f8c7 Expose max total num tokens from Runtime & Engine API (#2092) Henry Hyeonmok Ko 2024-11-22 15:10:10 -08:00
• 72f87b723b feat(pre-commit): trim unnecessary notebook metadata from git history (#2127) Xuehai Pan 2024-11-23 05:04:51 +08:00
• 62a4a339eb docs: fix module docstrings and copyright headers (#2077) Xuehai Pan 2024-11-22 22:16:53 +08:00
• 2797bc3422 fix: add xgrammar dependency (#2126) Yineng Zhang 2024-11-22 20:53:11 +08:00
• 9a00e6f453 chore: bump v0.3.6 (#2120) Yineng Zhang 2024-11-22 19:27:30 +08:00
• 4f8c3aeafc minor: update gsm8k threshold (#2125) Yineng Zhang 2024-11-22 19:23:58 +08:00
• 2369e88209 [minor] Clean up unused imports (#2122) Lianmin Zheng 2024-11-22 01:50:42 -08:00
• ad30d5cf9a Benchmark with Pytorch Profiler easily (#2110) bjmsong 2024-11-22 15:29:50 +08:00
• dfec7fca06 Rename sglang.bench_latency to sglang.bench_one_batch (#2118) Lianmin Zheng 2024-11-21 20:07:48 -08:00
• 8048c28c11 Fix #2037 - Context length check does not take into out pad tokens for visual models (#2106) Jake Poznanski 2024-11-21 19:05:41 -08:00
• 30af7dfb34 [router] add base_gpu_id server args & merged radix tree python reference (#2115) Byron Hsu 2024-11-21 17:13:33 -08:00
• f6f713797b Add support for Qwen2-VL-based embedding models (#2055) James Xu 2024-11-21 17:24:25 -05:00
• f35cb46cc3 ROCm: Fix MoE padding for none FP8 cases (#2111) HAI 2024-11-21 12:23:21 -08:00
• 7f8fcd39cd Turn off autotune for scaled mm for fp8 dynamic quant in torchao (#2116) Jerry Zhang 2024-11-21 12:19:49 -08:00
• 5c6a41facf Error out when torchao-config option is not recognized (#2107) Jerry Zhang 2024-11-20 17:37:28 -08:00
• 722530fa01 Enable overlap scheduler by default for the triton attention backend (#2105) Lianmin Zheng 2024-11-20 02:58:35 -08:00
• 56a347f7d3 Move test_session_id.py to playground (#2104) Lianmin Zheng 2024-11-20 01:28:27 -08:00
• 3295cd8af2 Allow skipping warmup in bench_offline_throughput.py (#2103) Lianmin Zheng 2024-11-20 01:25:21 -08:00
• 5942dfc00a [feat] Add session control (#2073) Ying Sheng 2024-11-20 00:36:53 -08:00
• 63a395b985 Update nightly-eval.yml (#2100) Lianmin Zheng 2024-11-19 22:15:02 -08:00
• 7d671e4ad2 Enable overlap by default (#2067) Lianmin Zheng 2024-11-19 22:07:58 -08:00
• 699384cb01 Set schedule policy more conservative for DP attention (#2096) Ke Bao 2024-11-20 12:57:18 +08:00
• ffd20fcd03 Make constrained decoding work for overlap scheduler (#2095) Lianmin Zheng 2024-11-19 15:04:43 -08:00
• 55bd97f3e5 minor: add dataset dump and questions shuffle (#2093) Yineng Zhang 2024-11-20 06:07:27 +08:00
• e57c3e12b8 Use native fp8 format on MI300X (#2094) HAI 2024-11-19 14:06:29 -08:00
• f239268fad minor: update gsm8k eval (#2091) Yineng Zhang 2024-11-19 20:36:55 +08:00