Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 8cc300f536 Fix router test (#4483) Byron Hsu 2025-03-16 22:49:47 -07:00
• 452db50808 Constraint Decoding: Set xgrammar as the default grammar backend (#4386) mlmz 2025-03-17 09:53:43 +08:00
• d1112d8548 Add endpoint for file support, purely to speed up processing of input_embeds. (#2797) Rin Intachuen 2025-03-17 08:30:37 +07:00
• 48efec7b05 Feature: support code completion (#3612) woodx 2025-03-17 09:26:19 +08:00
• 9b8333d992 [ROCm] enable moe topk softmax in amd (#4448) yiakwy-xpu-ml-framework-team 2025-03-17 09:16:55 +08:00
• f5bbf6037d Fix: Complete int32 to int64 conversion (#4465) Zhiqiang Xie 2025-03-16 18:14:27 -07:00
• 5cbd709ea1 Fix: modelscope env comment (#4474) huiwq1990 2025-03-17 09:11:33 +08:00
• 2e4a1e2d05 Initialize image processor for skip-tokenizer-init codepath (#4479) Yinghai Lu 2025-03-16 18:10:09 -07:00
• 9d02bb3e2a Urgent model support: support gemma-3-it (#4424) Mick 2025-03-17 08:37:32 +08:00
• 402db5c58c Benchmark: Statistical Analysis of the Output Stability of the Deepseek Model (#4202) ZelinTan 2025-03-17 08:32:57 +08:00
• 754a0e8278 Update CODEOWNERS (#4484) Lianmin Zheng 2025-03-16 17:10:15 -07:00
• 799fb5f455 400 on empty input_ids (#4481) Yinghai Lu 2025-03-16 14:01:23 -07:00
• 25e1816eff fix custom allreduce performance/accuracy problem (#4477) Yi Zhang 2025-03-17 03:16:30 +08:00
• a53fe428f9 Support FlashMLA backend (#4472) lukec 2025-03-17 00:07:06 +08:00
• 1b859295f4 [Eagle] Remove the greedy branch and some redundant code (#4363) Ying Sheng 2025-03-16 02:48:55 -07:00
• 9971dc2283 Revert "feat: Add FlashMLA submodule (#4449)" (#4470) Yineng Zhang 2025-03-16 01:30:05 -07:00
• 3db35c1af4 Release sgl-kernel v0.0.5.post2 (#4469) Lianmin Zheng 2025-03-16 01:01:53 -07:00
• 52a34d7448 Add greedy verification kernel (#4383) Ying Sheng 2025-03-16 00:58:26 -07:00
• 06d12b39d3 Remove filter for pr-tests (#4468) Lianmin Zheng 2025-03-16 00:57:26 -07:00
• c30976fb41 Fix finish step for pr tests and notebook tests (#4467) Lianmin Zheng 2025-03-16 00:52:06 -07:00
• 1a3fa75f2f [Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466) JieXin Liang 2025-03-16 15:02:47 +08:00
• 81f431eded feat: Add FlashMLA submodule (#4449) Shi Shuai 2025-03-16 14:30:25 +08:00
• 65b7c9b78f cleanup deps 2/n (#4464) Yineng Zhang 2025-03-15 23:06:17 -07:00
• 2c4f5ccac1 Fix minor style (#4460) Lianmin Zheng 2025-03-15 21:51:12 -07:00
• 158430473e Fix typos (#4368) Wang Ran (汪然) 2025-03-16 12:27:58 +08:00
• 8ec2ce0726 perf: update fused moe config (#4459) Mick 2025-03-16 12:23:57 +08:00
• 1fd0cf8a7b Update comment in qwen2.py (#4447) Michael Feil 2025-03-15 21:14:29 -07:00
• bf63ee54ed Auto-detect device if not specified in server arguments. (#4423) vikram singh shekhawat 2025-03-16 09:43:51 +05:30
• 22c96f78a6 typos: Update sampling_params.md (#4391) Wang Ran (汪然) 2025-03-16 07:40:18 +08:00
• 2892b9bb97 bugfix: Update sampling_params.py (#4413) Wang Ran (汪然) 2025-03-16 07:39:19 +08:00
• 470b474075 Update bench_serving.py (#4454) Xu Song 2025-03-16 07:33:58 +08:00
• 26c372c13c docs: Add Llama 3.3 to supported models (#4453) 江家瑋 2025-03-16 07:33:43 +08:00
• 86d9baedc2 [Fix] Fix errors when using the device except cuda. (#4455) Chen Shengzhi 2025-03-16 07:33:00 +08:00
• 21d485f835 Fix test_create_kvindices unit test (#4452) lukec 2025-03-16 07:01:04 +08:00
• 035ac2ab74 ci: update transformers==4.48.3 (#4451) Mick 2025-03-16 04:27:26 +08:00
• e1a5e7e47d docs: hot fix torch compile cache (#4442) Chayenne 2025-03-14 19:05:59 -07:00
• ad1ae7f7cd use topk_softmax with sgl-kernel (#4439) Yineng Zhang 2025-03-14 15:59:06 -07:00
• e73167ade3 Fix maximum recursion depth triggered on exception exit (#4438) Lianmin Zheng 2025-03-14 15:12:26 -07:00
• 862fe52241 bump v0.0.5.post1 (#4437) Yineng Zhang 2025-03-14 15:00:26 -07:00
• 61e4433caf Add moe topk softmax templated from vllm (#4302) Qingquan Song 2025-03-14 12:03:33 -07:00
• 660305c38a [Doc] fix wrong flag in deepseek documentation (#4427) Zhan Lu 2025-03-14 18:30:55 +00:00
• 642ab418f3 [bug] fix duplicate variable MAX_PIXELS in qwen_vl.py (#4419) Baoyuan Qi 2025-03-14 16:28:25 +08:00
• 1ce4878d31 feat(remote_model): support variable remote backend for model loader (#3964) wangyu 2025-03-14 15:40:44 +08:00
• 977d7cd26a cleanup deps 1/n (#4400) Yineng Zhang 2025-03-14 00:00:33 -07:00
• 0e0ec70200 Hierarchical Caching supports MLA (#4009) Lu Changqi 2025-03-14 11:42:14 +08:00
• bb37855653 Update CODEOWNERS (#4403) Lianmin Zheng 2025-03-13 17:54:40 -07:00
• ba80c102f9 bump v0.4.4.post1 (#4402) Yineng Zhang 2025-03-13 17:53:46 -07:00
• fbdb50501f Hot fix for hicache with new page aligned radixtree (#4397) Zhiqiang Xie 2025-03-13 15:50:49 -07:00
• f0afaf5289 Add a dummy grok test case (#4399) Lianmin Zheng 2025-03-13 15:29:48 -07:00
• 85d2365d33 Fix the output of hidden states after HTTP requests (#4269) Qiaolin Yu 2025-03-13 17:54:06 -04:00
• 5fe79605a8 Fix Llama3.3 tool call support (#4320) Chang Su 2025-03-13 14:01:41 -07:00
• c6d7f8d370 Add some fused elementwise kernels for grok-1 (#4398) Lianmin Zheng 2025-03-13 13:39:10 -07:00
• a5a892ffd3 Fix auto merge & add back get_flat_data_by_layer (#4393) Lianmin Zheng 2025-03-13 08:46:25 -07:00
• 8e66fbecee Improve DP attention (#4390) Lianmin Zheng 2025-03-13 08:23:56 -07:00
• f141298a3c Update ci_install_dependency.sh to use accelerate 1.4.0 (#4392) Lianmin Zheng 2025-03-13 07:16:11 -07:00
• 4fea040ca1 Fix a regression introduced by overlapping KV cache writing (#4375) Lianmin Zheng 2025-03-13 03:49:05 -07:00
• 6aaeb84872 chore: bump v0.4.4 (#4041) Yineng Zhang 2025-03-13 02:49:58 -07:00
• 3623b6a7f5 upgrade sgl-kernel 0.0.5 (#4381) Yineng Zhang 2025-03-13 02:37:56 -07:00
• 4ff1264201 Update pyproject.toml Yineng Zhang 2025-03-13 02:16:51 -07:00
• 2a4cbad8e9 bump 0.0.5 sgl-kernel (#4377) Yineng Zhang 2025-03-13 02:08:35 -07:00
• 2937387a50 fix accuracy issue (#4376) Yineng Zhang 2025-03-13 02:06:22 -07:00
• cf721fdece Update grafana.json (#4374) yuhui 2025-03-13 16:31:33 +08:00
• 45de89719c Revert "[XPU][CPU] Enable the native path of DeepSeek" (#4367) Lianmin Zheng 2025-03-12 23:45:52 -07:00
• 71046fcd71 [XPU][CPU] Enable the native path of DeepSeek (#4086) Meng, Hengyu 2025-03-13 13:26:29 +08:00
• c76040e31b Support page size > 1 (#4356) Lianmin Zheng 2025-03-12 22:22:39 -07:00
• 2f6bacee03 [moe] fix: correct the cache size in the last chunk (#3679) Cheng Wan 2025-03-13 01:22:13 -04:00
• 4014804157 Ensure Usage Data in Streaming Responses Aligns with vLLM’s Implementation (#3814) Wen Sun 2025-03-13 13:12:55 +08:00
• ad46550d25 [Doc] Fix typo in backend/sampling_params (#3835) yang_zcybb 2025-03-13 13:12:14 +08:00
• 14344caa38 [docs] Update outdated description about torch.compile (#3844) Jun Liu 2025-03-13 14:09:38 +09:00
• f7f88b706c HotFix: json serialization error when using OAI v1/batches endpoint with logprobs (#3896) David Carreto Fidalgo 2025-03-13 06:04:29 +01:00
• 18c27131f5 [tools] add fp8 max/min constant in utils (#3959) yiakwy-xpu-ml-framework-team 2025-03-13 12:44:55 +08:00
• ccdd10c84b Move aiohttp into public dependencies (#3980) YR Chen 2025-03-13 12:42:57 +08:00
• 76f6c0ebf9 Add device detection and count functions to utils. (#3962) vikram singh shekhawat 2025-03-13 10:11:50 +05:30
• 959a3143fc example: add async offline inference demo (#3961) Chitsing KUI 2025-03-13 12:41:21 +08:00
• 6412c5e493 Avoid duplicated request ids in batch APIs (#4026) Conghui Tan 2025-03-13 12:38:17 +08:00
• 0c02086015 add INT8 example into dsv3 README (#4079) laixin 2025-03-13 12:37:30 +08:00
• 85ef7f64e4 [FIX] fix incorrect output when enable both deepgemm and torch compile (#4359) AniZpZ 2025-03-13 12:34:09 +08:00
• f1cf6eefbe [Fix] Check the device backend before calling empty_cache function (#4212) Chen Shengzhi 2025-03-13 12:28:48 +08:00
• 0a59a4657a Fix the doc of FR-Spec (#4295) William 2025-03-13 12:22:50 +08:00
• aff79f101f simple bugfix (#4342) Wang Ran (汪然) 2025-03-13 12:20:18 +08:00
• 016033188c docs: add parameter --log-requests-level (#4335) Peter Pan 2025-03-13 12:19:37 +08:00
• 56c39a05a2 Remove the choices in --speculative-eagle-topk argument (#4329) William 2025-03-13 12:19:16 +08:00
• 4068e01292 Fix per token fp8 quant precision (#4362) Qingquan Song 2025-03-12 21:19:05 -07:00
• 817d43705c feat: support ep size < 32 for sgl kernel (#4348) Shi Shuai 2025-03-13 11:50:46 +08:00
• c550e52f8b Fix scheduler proctitle suffix is ​​None (#4326) 文峰 2025-03-13 10:29:35 +08:00
• e35a93fa8a Move output processing logic from scheduler.py into a separate file (#4354) Lianmin Zheng 2025-03-12 16:21:49 -07:00
• 2c3656f276 [Fix Doc.] Enable internal forwarding when starting the router (#4355) shizhediao 2025-03-12 15:53:26 -07:00
• d40ee62b5d Update nightly tests (#4352) Lianmin Zheng 2025-03-12 15:36:13 -07:00
• 91b19949d7 typo: Update http_server.py (#4350) Wang Ran (汪然) 2025-03-13 06:05:30 +08:00
• 7c86671131 Support Blackwell Block Scale FP8 Gemm (#4278) Elfie Guo 2025-03-12 14:17:11 -07:00
• 10b544ae9b Hierarchical Caching Refactoring and Fixing TP issue (#4082) Zhiqiang Xie 2025-03-12 11:22:35 -07:00
• 01090e8ac3 model: Support Janus-pro (#3203) Mick 2025-03-13 02:02:11 +08:00
• 6f43a9b9f4 remove the unused readline dependency from the Qwen2 model implementa… (#4340) yych0745 2025-03-12 17:47:27 +08:00
• 0540fef7a1 [Fix] fix _yarn_linear_ramp_mask with device parameter (#4337) JieXin Liang 2025-03-12 17:28:19 +08:00
• 481f608b8e Add INT8 support MTP NextN function (#3911) lambert0312 2025-03-12 16:37:16 +08:00
• ed91561f79 upgrade sgl-kernel 0.0.4.post3 (#4334) Yineng Zhang 2025-03-12 01:36:41 -07:00
• 6e7239f912 release 0.0.4.post3 sgl-kernel (#4331) Yineng Zhang 2025-03-12 01:05:16 -07:00
• 0a3960f21f fix awq_dequantize (#4333) Yineng Zhang 2025-03-12 01:04:38 -07:00
• 07f944631e Add awq dequantize kernel to sgl with 1x to 3x speedup (#4104) Rex 2025-03-12 00:10:02 -07:00
• e0917e6bd0 Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215) Stefan He 2025-03-12 00:08:03 -07:00