Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 153b414e83 minor: sync flashinfer and add turbomind as 3rdparty (#3105) Yineng Zhang 2025-01-24 19:22:39 +08:00
• 6619f48e18 Fix cu118 group gemm compile issue (#3097) Ke Bao 2025-01-24 15:19:09 +08:00
• 3ed0a547b2 [router] Fix twine uploading (#3095) Byron Hsu 2025-01-23 21:01:01 -08:00
• 8d8ef8497e bump router to 0.1.4 (#3094) Byron Hsu 2025-01-23 20:32:43 -08:00
• 9a0cc2e90e [router] Forward all request headers from router to workers (#3070) Byron Hsu 2025-01-23 20:30:31 -08:00
• 7bad7e75bf Add shapes for int8 gemm benchmark (#3093) Ke Bao 2025-01-24 12:27:30 +08:00
• 1c4e0d2445 Docs: Update doc for server arguments (#2742) simveit 2025-01-23 20:32:05 +01:00
• 54bac8af0b chore: bump sgl-kernel 0.0.2.post16 (#3087) Yineng Zhang 2025-01-24 01:57:48 +08:00
• 5de4051bcf feat: integrate sampling kernels into sgl-kernel (#3086) Yineng Zhang 2025-01-24 01:54:47 +08:00
• e0cd65c2b6 [hotfix] fix test_sampling_scaling_penalties.py ci test (#3084) Xiaoyu Zhang 2025-01-24 00:33:59 +08:00
• f1b6861828 use flashinfer vec_dtypes in sgl_kernel (#3083) Xiaoyu Zhang 2025-01-23 22:19:04 +08:00
• 0da0989ad4 sync flashinfer and update sgl-kernel tests (#3081) Yineng Zhang 2025-01-23 21:13:55 +08:00
• 07a22cbba3 use env variable to control the build conf on the CPU build node (#3080) Yineng Zhang 2025-01-23 20:46:49 +08:00
• 3d0bfa3e17 update version setup for sgl-kernel (#3079) Yineng Zhang 2025-01-23 19:45:25 +08:00
• 1f6cf0d4b9 fix build error for sgl-kernel (#3078) Yineng Zhang 2025-01-23 19:16:35 +08:00
• 553f5a3ffe Remove torch dependency in sgl-kernel (#3074) Lianmin Zheng 2025-01-23 01:23:37 -08:00
• ac2dc35d0e support lightning_attention_decode in sgl-kernel for MiniMax-Text-01 (#3030) Xiaoyu Zhang 2025-01-23 15:29:20 +08:00
• 3e032c07cc use v0.6.4.post1 for sgl-kernel ci (#3071) Yineng Zhang 2025-01-23 14:19:38 +08:00
• 44e12ce463 docs: update developer guide for sgl-kernel (#3069) Yineng Zhang 2025-01-23 14:08:25 +08:00
• a547aad61f docs: add developer guide for sgl-kernel (#3068) Yineng Zhang 2025-01-23 13:47:53 +08:00
• ea535dc574 Revert "disable custom allreduce on HIP" (#3067) Lianmin Zheng 2025-01-22 21:33:35 -08:00
• 862bcff833 Support loading of larger models with on-the-fly quantization (#3061) Ke Wen 2025-01-22 21:33:17 -08:00
• 8b84e69f25 Fix tp token sync for dp attention (#3062) Lianmin Zheng 2025-01-22 18:51:40 -08:00
• 5de50653cd [router] make error actionable (#3063) Byron Hsu 2025-01-22 17:56:21 -08:00
• c0bf9bf15c [devcontainer] add non-root user (#2989) Byron Hsu 2025-01-22 17:47:54 -08:00
• 022614d26e Add some flags to allow sync token ids across TP ranks (#3060) Lianmin Zheng 2025-01-22 15:05:51 -08:00
• b8ab989ff4 Fix the FP8 E4M3 parsing offline scales failure bug (#3045) lukec 2025-01-23 06:19:33 +08:00
• b3393e941f [Doc] Update doc of profiling with PyTorch Profiler (#3038) Baizhou Zhang 2025-01-22 14:17:26 -08:00
• ddc2001fb0 disable custom allreduce on HIP (#3058) Hui Liu 2025-01-22 13:57:22 -08:00
• 806a3002c1 add notice about flashinfer in sgl-kernel (#3057) Yineng Zhang 2025-01-23 02:47:36 +08:00
• 0d2148efaa fix rotary_embedding rope_scaling for phi (#3055) nstream-ai-devx 2025-01-22 23:45:32 +05:30
• bf669606eb feat: integrate bmm_fp8 kernel into sgl-kernel (#3056) Yineng Zhang 2025-01-23 00:39:38 +08:00
• b2bd8f444c minor: update header and use pytest (#3054) Yineng Zhang 2025-01-22 23:45:18 +08:00
• 9d9b482a39 feat: integrate activation kernels into sgl-kernel (#3053) Yineng Zhang 2025-01-22 23:25:45 +08:00
• 7353fb9b97 feat: integrate norm kernels into sgl-kernel (#3052) Yineng Zhang 2025-01-22 21:32:48 +08:00
• bcda0c9ee6 sync the upstream updates of flashinfer (#3051) Yineng Zhang 2025-01-22 20:33:13 +08:00
• 9f8f2c7f74 update norm cu (#3048) Yineng Zhang 2025-01-22 18:58:44 +08:00
• 6fc37bd8ee Fix sgl-kernel compile for sm80 (#3046) Ke Bao 2025-01-22 16:49:08 +08:00
• 3d8f1c9bcf Use int64 as indices for set_kv_buffer (#3039) Lianmin Zheng 2025-01-21 19:46:09 -08:00
• a42213dbd4 fix pr-test-sgl-kernel (#3036) Yineng Zhang 2025-01-22 00:56:42 +08:00
• 0ac019f171 Support sm90 Int8 gemm (#3035) Ke Bao 2025-01-21 22:21:54 +08:00
• 5a0d680a14 feat: add flashinfer as 3rdparty and use rmsnorm as example (#3033) Yineng Zhang 2025-01-21 20:44:49 +08:00
• a4331cd260 Add accuracy and latency tests of eagle into CI (#3027) Lianmin Zheng 2025-01-21 02:55:14 -08:00
• ec1c21cdc4 upgrade torch version for sgl-kernel (#3026) Yineng Zhang 2025-01-21 14:32:08 +08:00
• 6c856b4f3a minor: update Makefile for sgl-kernel (#3025) Yineng Zhang 2025-01-21 13:08:15 +08:00
• 287d07a669 Misc fixes for eagle (flush_cache, CPU overhead) (#3014) Lianmin Zheng 2025-01-20 20:25:13 -08:00
• d2571dd5c7 Enable Cohere2 Models (#3018) Hui Liu 2025-01-20 19:21:41 -08:00
• b730aa6b9e [EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (#2939) 996_icu 2025-01-21 09:46:43 +08:00
• 60b2a44a80 Fix flaky tests in test_programs.py (#3022) Lianmin Zheng 2025-01-20 16:50:39 -08:00
• 949b3fbfce [Doc] Update doc of custom logit processor (#3021) Hongpeng Guo 2025-01-20 16:50:25 -08:00
• da4e8b3892 enable kv_scale remap (#3017) Hui Liu 2025-01-20 14:40:45 -08:00
• af6c5357d5 deepseek v3 and r1 chat template (#3015) Enrique Shockwave 2025-01-20 22:40:12 +00:00
• 3ad4cd4915 bump router to 0.1.3 (#3020) Byron Hsu 2025-01-20 14:38:06 -08:00
• 3a8428ecaa [router] Expose worker startup interval (#3019) Byron Hsu 2025-01-20 14:36:54 -08:00
• 0311ce8e1c [router] Expose worker startup secs & Return error instead of panic for router init (#3016) Byron Hsu 2025-01-20 12:45:13 -08:00
• 5dfcacfcb1 Add compile flags for cutlass 3.x (#3013) Ke Bao 2025-01-21 00:04:12 +08:00
• 41a0ccd4f1 Add clang-format check to sgl-kernel ci (#3012) Ke Bao 2025-01-20 23:22:19 +08:00
• e94fb7cb10 chore: bump v0.4.1.post7 (#3009) Yineng Zhang 2025-01-20 21:50:55 +08:00
• b5caa22dfb [kernel] port rope cuda kernel to sgl-kernel (#2993) Byron Hsu 2025-01-20 04:58:51 -08:00
• 73401fd016 Sync distributed package from vllm 0.6.4.post1 (#3010) Lianmin Zheng 2025-01-20 04:57:14 -08:00
• 89cd923581 Roll back to use vllm custom allreduce (#3006) Lianmin Zheng 2025-01-20 04:03:15 -08:00
• dc1881326f Fix perf regression on small batch sizes (#3008) Lianmin Zheng 2025-01-20 03:39:49 -08:00
• 10bfce71b3 fix moe align blocks benchmark (#3003) yiakwy-xpu-ml-framework-team 2025-01-20 19:33:29 +08:00
• 583697cd71 [Enhancement] Custom Logit Processor Improvement (#2998) Hongpeng Guo 2025-01-20 02:00:35 -08:00
• 2584f6d944 Docs: Add Performance Demonstaration for DPA (#3005) Chayenne 2025-01-20 01:00:52 -08:00
• 51e87f6f21 Skip flaky custom_logit_processor tests (#3004) Lianmin Zheng 2025-01-20 00:28:47 -08:00
• 09bcbe0123 Update TypeBasedDispatcher and balance CI tests (#3001) Lianmin Zheng 2025-01-19 23:37:27 -08:00
• 03464890e0 Separate two entry points: Engine and HTTP server (#2996) Lianmin Zheng 2025-01-19 22:09:24 -08:00
• 44a9669770 keep rotary_embedding only (#2997) Yineng Zhang 2025-01-20 13:21:36 +08:00
• 1a820e38a2 Remove dependency of pynvml on ROCm (#2995) Chaitanya Sri Krishna Lolla 2025-01-20 10:30:35 +05:30
• 0ffcfdf474 Docs: Only use X-Grammar in structed output (#2991) Chayenne 2025-01-19 20:22:47 -08:00
• cd493b5afc Improve metrics, logging, and importing orders (#2992) Lianmin Zheng 2025-01-19 18:36:59 -08:00
• 61f42b5732 Move sgl.Runtime under sglang/lang (#2990) Lianmin Zheng 2025-01-19 17:10:29 -08:00
• e403d23757 [Feature] Add sampler custom logits processor (#2396) Hongpeng Guo 2025-01-19 14:46:53 -08:00
• 3bcf5ecea7 support regex in xgrammar backend (#2983) Enrique Shockwave 2025-01-19 20:34:41 +00:00
• 2c05f81f15 fix custom op version compatibility (#2988) Yineng Zhang 2025-01-20 04:21:29 +08:00
• d77caa2b75 [#2812] Make the decode status dict capcity adjustable by a CLI param (#2839) Seungduk Kim 2025-01-20 04:36:53 +09:00
• 8b6a4486ec fix missing revision arg when loading tokenizer (#2982) giorgiopiatti-dfinity 2025-01-19 20:36:07 +01:00
• a69cb5cff7 cleanup unused header in sgl_kernel (#2986) Yineng Zhang 2025-01-20 00:44:49 +08:00
• def5c31873 docs: update supported_models (#2987) Yineng Zhang 2025-01-20 00:44:30 +08:00
• 3fc2b62589 update docker dev image (#2985) Yineng Zhang 2025-01-19 23:45:39 +08:00
• 6ada05d0ed feat: check for is_cuda for sgl_kernel import (#2984) Yineng Zhang 2025-01-19 23:33:04 +08:00
• 24cafe3177 add config to swtich from vllm custom allreduce to sgl_kernel custom allreduce (#2981) yizhang2077 2025-01-19 22:30:38 +08:00
• 5a176c92df fix deepseek v2 with cpu device (#2975) Yineng Zhang 2025-01-19 21:33:27 +08:00
• 4719c1d04a [router] Fix sgl router path for release (#2980) Byron Hsu 2025-01-19 17:11:06 +08:00
• ef18b0eda2 [router] Allow empty worker list for sglang.launch_router (#2979) Byron Hsu 2025-01-19 17:05:23 +08:00
• 53cc91e504 [devcontainer] Fix mount and GPU & Support rust dev (#2978) Byron Hsu 2025-01-19 16:34:01 +08:00
• d33cbb7e58 remove cub and add cccl (#2976) Yineng Zhang 2025-01-19 15:51:27 +08:00
• 23196d5254 Simplify logits processor (#2974) Lianmin Zheng 2025-01-18 23:03:49 -08:00
• 93b77c8e8a Fix the request loggings to make it fully able to be easily replayed (#2973) Lianmin Zheng 2025-01-18 21:45:00 -08:00
• 7906d1d298 Remove the unused write_with_records (#2972) Lianmin Zheng 2025-01-18 20:20:23 -08:00
• 81d27c8e31 Refactor to add TypeBasedDispatcher to simplify dispatching (#2958) fzyzcjy 2025-01-19 12:13:27 +08:00
• 4d4cdb3fe7 Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956) Chang Su 2025-01-18 19:37:30 -08:00
• 2bd18e2d76 Memory pool: Minor optimize to avoid to (#2901) Yang Zheng 2025-01-19 11:35:12 +08:00
• 83452dbb4a fix file name spelling mistake and useless variable in minmax-text-01-lightning_attention (#2971) Xiaoyu Zhang 2025-01-19 10:56:13 +08:00
• 3d93f84a00 [Feature] Support minicpmv v2.6 (#2785) Mick 2025-01-19 06:14:19 +08:00
• c2f212d672 optimize MiniMax-Text-01 lightning_attn_decode triton (#2966) Xiaoyu Zhang 2025-01-18 23:41:01 +08:00
• e2cdc8a5b5 upgrade cutlass v3.7.0 (#2967) Yineng Zhang 2025-01-18 23:37:42 +08:00
• 2add697d7a feat: remove vllm get_rope (#2964) Yineng Zhang 2025-01-18 19:38:01 +08:00
• 6f98c586bd fix sgl-kernel setup.py (#2963) lukec 2025-01-18 18:50:37 +08:00