Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 656dcc1a99 Remove fp8 monkey patch (#2960) Ke Bao 2025-01-18 15:00:29 +08:00
• 8af7048dcf Query remaining memory dynamically for PrefillAdder (#2941) Zhiqiang Xie 2025-01-17 20:20:26 -08:00
• d3024f4fc8 support e4m3 kvcache in qwen2 & add kv scaling facotr json (#2894) bjmsong 2025-01-18 11:43:22 +08:00
• 13387e6b7a Multi-turn benchmark for hierarchical caching (#2942) Zhiqiang Xie 2025-01-17 16:17:24 -08:00
• 120c3634ef Fix Llama-3.1-405B References Docs (#2944) Wen Sun 2025-01-18 06:46:38 +08:00
• 78e5b22f29 feat: use get_rope for gemma2 (#2954) Yineng Zhang 2025-01-18 02:57:18 +08:00
• 7a15e9ad36 cleanup models unused import 2/n (#2952) Yineng Zhang 2025-01-18 01:09:19 +08:00
• dc2ac0cbdb Update pr template (#2951) Ke Bao 2025-01-18 00:44:16 +08:00
• d47c5101f1 Add ut for qwen model (#2947) Ke Bao 2025-01-18 00:03:54 +08:00
• 033c715b46 cleanup models dependencies 1/n (#2948) Yineng Zhang 2025-01-17 23:46:48 +08:00
• d06c1ab587 update ci install dependency (#2949) Yineng Zhang 2025-01-17 23:42:23 +08:00
• c5644cace9 docs: add Cursor for adoption and sponsorship (#2950) Yineng Zhang 2025-01-17 23:41:57 +08:00
• 53e6552fed Fix qwen accuracy issue (#2945) Ke Bao 2025-01-17 22:35:26 +08:00
• 5dc54f1a62 feat: remove vllm distributed (#2907) Yineng Zhang 2025-01-17 22:31:51 +08:00
• f3e9b4894b Fix sgl-kernel ci (#2938) Ke Bao 2025-01-17 17:26:21 +08:00
• 6a7973add8 Update release-docs.yml (#2937) Lianmin Zheng 2025-01-17 00:36:40 -08:00
• 63051738a9 Enable CPU device on SGLang (#2806) Chunyuan WU 2025-01-17 13:22:53 +08:00
• a8ccacc8b8 [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) Chang Su 2025-01-16 14:51:19 -08:00
• 0427416b59 Fix zmq binding (#2930) Lianmin Zheng 2025-01-16 14:36:07 -08:00
• bf3edc2c60 Docs: Update pull_request_template.md (#2928) Chayenne 2025-01-16 13:04:11 -08:00
• 78e974b2a5 [kernel] MiniMax-Text-01 decode lightning_attn with triton (#2920) Xiaoyu Zhang 2025-01-17 04:51:38 +08:00
• bc6915e3b9 Improve type annotation and styles (#2926) Lianmin Zheng 2025-01-16 12:51:11 -08:00
• a883f0790d Update release-docker-amd.yml to run on amd docker runner. (#2927) saienduri 2025-01-16 12:42:29 -08:00
• 8b6ce52e92 Support multi-node DP attention (#2925) Lianmin Zheng 2025-01-16 11:15:00 -08:00
• 58f3f2b840 Add CI for sgl-kernel (#2924) Ke Bao 2025-01-17 01:26:51 +08:00
• 93d690617e Simplify the process launch code in server.py (#2923) Lianmin Zheng 2025-01-16 07:52:17 -08:00
• e00e5385e0 add profiling to bench_one_batch script (#2821) Yun Dai 2025-01-16 07:24:24 -08:00
• a2f602b541 fixed lm_head.weight error for quantized qwen (#2910) Rin Intachuen 2025-01-16 21:51:43 +07:00
• 8f2c522aba Improve benchmark scripts and error message printing (#2922) Lianmin Zheng 2025-01-16 06:24:31 -08:00
• 7596417732 minor: use bear for compilation database (#2919) Yineng Zhang 2025-01-16 18:39:11 +08:00
• 2dc957d421 fix setup for sgl kernel (#2917) Yineng Zhang 2025-01-16 18:17:34 +08:00
• bf8d07a6f9 feat: patch linear base (#2915) Yineng Zhang 2025-01-16 18:00:03 +08:00
• ab31793661 [kernel] MiniMax-Text-01 prefill lightning_attn with triton (#2911) Xiaoyu Zhang 2025-01-16 14:18:29 +08:00
• b7f3fec13c minor: rename bench for sgl kernel (#2909) Yineng Zhang 2025-01-16 05:55:43 +08:00
• 58f42b1dd8 minor: update pr test (#2908) Yineng Zhang 2025-01-16 05:51:49 +08:00
• 767c9dec03 adapt custom allreduce for tensorrt llm (#2511) yizhang2077 2025-01-16 04:57:35 +08:00
• a53454c55e fix: sgl-kernel link cuda (#2906) Yineng Zhang 2025-01-16 04:53:23 +08:00
• 6cb3974e77 optimize custom allreduce kernel (#2904) yizhang2077 2025-01-16 03:04:25 +08:00
• f65c13b559 Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902) Lianmin Zheng 2025-01-15 04:27:18 -08:00
• b803b395b7 Disable graceful shutdown of tokenizer manager when not in the main thread (#2872) Cody Yu 2025-01-15 03:29:33 -08:00
• bfbda62c8b Add ut for w8a8 int8 quantization (#2897) Ke Bao 2025-01-15 18:29:14 +08:00
• b3e99dfb22 chore: bump v0.4.1.post6 (#2899) Yineng Zhang 2025-01-15 16:23:42 +08:00
• f005758f2b introduce CUB in sgl-kernel (#2887) Xiaoyu Zhang 2025-01-14 19:48:59 +08:00
• f5c6c66794 feat: support internlm 3 dense (#2888) Yineng Zhang 2025-01-14 19:23:26 +08:00
• cc0485bef2 Support w8a8 int8 quantization config (#2881) Ke Bao 2025-01-14 17:07:49 +08:00
• b8cd09f27a update ROCm docker for layernorm kernel optimization (#2885) kk 2025-01-14 16:59:43 +08:00
• c19d84829c Adjust flashinfer workspace size for Qwen2 models (#2879) Ke Bao 2025-01-14 13:34:22 +08:00
• 80002562a8 docs: update README (#2878) Yineng Zhang 2025-01-14 12:48:17 +08:00
• 46d4431889 Add a new api configure_logging to allow dumping the requests (#2875) Lianmin Zheng 2025-01-13 14:24:00 -08:00
• 923f518337 CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) fzyzcjy 2025-01-14 03:38:51 +08:00
• d08c77c434 Sampling penalties memory interface (#2870) Xiaoyu Zhang 2025-01-13 23:09:00 +08:00
• c1e097ca66 Revert "Dump requests to a folder" (#2869) Lianmin Zheng 2025-01-13 06:21:25 -08:00
• 6ec75e626d add qwen2 eagle model (#2863) Lzhang-hub 2025-01-13 21:29:33 +08:00
• d855653bd4 minor: fix release docs (#2868) Yineng Zhang 2025-01-13 21:18:39 +08:00
• 336ff5b9f5 Fix typos in io_struct.py (#2867) Lianmin Zheng 2025-01-13 05:13:02 -08:00
• 3b141e1509 Dump requests (#2862) Lianmin Zheng 2025-01-13 04:51:56 -08:00
• 6249e4a19e Revert "Integration of TurboMind AWQ" (#2866) Lianmin Zheng 2025-01-13 04:44:39 -08:00
• f3516c2894 Fix quant kernel accuracy issue (#2865) Ke Bao 2025-01-13 20:32:17 +08:00
• 17de02f98d Integration of TurboMind AWQ (#2828) bjmsong 2025-01-13 20:14:16 +08:00
• 51ab3ccf47 Collect more metrics: num_requests_total (#2859) Lianmin Zheng 2025-01-13 03:57:39 -08:00
• 67008f4b32 Use only one GPU for MLA CI tests (#2858) Lianmin Zheng 2025-01-13 03:55:33 -08:00
• 4536d72446 minor: use ubuntu-latest instead of self-hosted runner for amd build (#2861) Yineng Zhang 2025-01-13 18:58:56 +08:00
• 41d7e5b7e6 docs: update link (#2857) Yineng Zhang 2025-01-13 18:40:48 +08:00
• 20a9f5dfe0 fix: not delete CNAME (#2860) Yineng Zhang 2025-01-13 18:36:40 +08:00
• 42f3909963 Unify sglang coding style (#2856) kk 2025-01-13 18:12:44 +08:00
• 72c7776355 Fix linear.py and improve weight loading (#2851) Lianmin Zheng 2025-01-13 01:39:14 -08:00
• 4093aa4660 [Fix]eagle2 health_generate is first request,apiserver will core (#2853) justdoit 2025-01-13 17:01:21 +08:00
• e808c1df3e Integrate ROCm ater package for ck moe function feasibility (#2854) kk 2025-01-13 16:23:07 +08:00
• a18ab81ddd Update base image for ROCm (#2852) sogalin 2025-01-13 14:39:44 +08:00
• 0bb0f76311 Support FP8 E4M3 KV Cache (#2786) bjmsong 2025-01-13 13:17:11 +08:00
• 85b2e05770 Add int8 quant kernel (#2848) Ke Bao 2025-01-13 13:16:58 +08:00
• a879c2fb4c fix sgl-kernel build (#2850) Yineng Zhang 2025-01-13 12:27:17 +08:00
• e2b16c4716 add sampling_scaling_penalties kernel (#2846) Xiaoyu Zhang 2025-01-13 11:38:17 +08:00
• c4f9707e16 Improve: Token-In Token-Out Usage for RLHF (#2843) Shi Shuai 2025-01-11 23:14:26 +00:00
• 197cbf9bab docs: update README (#2841) Yineng Zhang 2025-01-11 23:11:38 +08:00
• f624901cdd chore: bump v0.4.1.post5 (#2840) Yineng Zhang 2025-01-11 23:10:02 +08:00
• f0e15dc6ab [HotFix] fix fp8 scale load failed in tp>1 (#2837) Xiaoyu Zhang 2025-01-11 14:34:26 +08:00
• f1769586d6 Update threshold in test_nightly_gsm8k_eval.py (#2836) Lianmin Zheng 2025-01-10 20:37:34 -08:00
• 5d6e9467d4 Cache controller for hierarchical caching (#2804) Zhiqiang Xie 2025-01-10 20:22:01 -08:00
• a47bf39123 [Eagle2] Fix multiple concurrent request crashes (#2730) justdoit 2025-01-11 06:00:43 +08:00
• b170646991 Fix port number overflow (#2826) TianYu GUO 2025-01-11 05:44:32 +08:00
• 5413ec2bbe [Bugfix] Fix bug in fork logic caused by null text_ (#2835) Muqi Li 2025-01-11 05:37:00 +08:00
• f290bd4332 [Bugfix] Fix embedding model hangs with --enable-metrics (#2822) Chang Su 2025-01-10 13:14:51 -08:00
• 8f15789314 Add more metrics to serving benchmark. (#2819) Pratyush Patel 2025-01-10 07:30:44 -08:00
• 2db03a04ca Update README.md (#2833) Lianmin Zheng 2025-01-10 03:49:04 -08:00
• 5cc1170552 Doc: add block-wise FP8 in dpsk model reference (#2830) Chayenne 2025-01-10 00:26:59 -08:00
• 11fffbc95a [Doc]: Deepseek reference docs (#2787) Xiaotong Jiang 2025-01-09 13:43:12 -08:00
• 4f077c01b8 minor: support specifying local dataset path for gsm8k and hellaswag (#2816) sleepcoo 2025-01-09 22:24:42 +08:00
• 679c3bcacf Fix typo in cuda_graph_bs (#2813) Lianmin Zheng 2025-01-09 03:03:24 -08:00
• 656aed58c6 Remove vllm dependency in model config (#2809) Yunmeng 2025-01-09 17:51:56 +08:00
• b5fb4ef58a Update modelopt config and fix running issue (#2792) Ke Bao 2025-01-08 18:04:30 +08:00
• 2e6346fc2e Docs：Update the style of llma 3.1 405B docs (#2789) Chayenne 2025-01-08 01:07:54 -08:00
• 977f785dad Docs: Rewrite docs for LLama 405B and ModelSpace (#2773) mlmz 2025-01-08 16:02:59 +08:00
• 8a6906127a Improve linear.py to load sharded weights & remove the dependency of Parameters from vllm (#2784) Lianmin Zheng 2025-01-07 23:29:10 -08:00
• 694e41925e [eagle2] fix end check when target model verify (#2723) JJJJOHNSON 2025-01-08 13:46:02 +08:00
• b22f3f6475 Fix nightly accuracy tests (#2780) Lianmin Zheng 2025-01-07 21:02:35 -08:00
• 6fb5768372 Disable math eval on nightly CI temporarily (#2779) Lianmin Zheng 2025-01-07 18:17:34 -08:00
• 51caee740f Host memory pool for hierarchical caching (#2771) Zhiqiang Xie 2025-01-07 13:38:37 -08:00
• 58f9060efe Update int8 gemm config (#2774) Ke Bao 2025-01-07 19:47:37 +08:00
• bdc1acf6cd Misc fix for min_p_sampling, --cuda-graph-bs (#2761) Lianmin Zheng 2025-01-07 02:52:53 -08:00