Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 0736b27020 [Minor] Improve the code style in TokenizerManager (#767) Lianmin Zheng 2024-07-27 05:05:15 -07:00
• 3fdab91912 Fix TransformerTokenizer init for chatglm2 & 3 (#761) Ke Bao 2024-07-27 17:44:46 +08:00
• ba29504b21 Update supported models (#763) Liangsheng Yin 2024-07-26 22:53:53 -07:00
• a72342f180 fix: not run workflows on fork repo (#762) Yineng Zhang 2024-07-27 14:51:33 +10:00
• c3c74bf874 docs: update model support (#760) Yineng Zhang 2024-07-27 14:07:37 +10:00
• d9fccfefe2 Fix context length (#757) Liangsheng Yin 2024-07-26 18:13:13 -07:00
• 679ebcbbdc Deepseek v2 support (#693) Liangsheng Yin 2024-07-26 17:10:07 -07:00
• 5bd06b4599 fix: use REPO_TOKEN (#755) Yineng Zhang 2024-07-27 05:56:30 +10:00
• 9a61182732 fix: add release tag workflow (#754) Yineng Zhang 2024-07-27 05:48:38 +10:00
• eeb2482186 feat: add release tag workflow (#753) Yineng Zhang 2024-07-27 05:37:02 +10:00
• 3e455b016e misc: replace deprecated variable HUGGING_FACE_HUB_TOKEN with HF_TOKEN (#752) Yineng Zhang 2024-07-27 04:19:30 +10:00
• 8628ab9c8b feat: add docker workflow (#751) Yineng Zhang 2024-07-27 03:54:51 +10:00
• 1b77670f39 chore: bump v0.2.1 (#740) Yineng Zhang 2024-07-26 21:27:41 +10:00
• 768e05d08f fix benchmark (#743) Yineng Zhang 2024-07-26 21:26:13 +10:00
• 01fbb11bb7 docs: fix typo (#742) Yineng Zhang 2024-07-26 21:05:53 +10:00
• 05d216da32 docs: add llama 3.1 405b instruction (#739) Yineng Zhang 2024-07-26 21:03:20 +10:00
• 6b32bb1c0b misc: format (#741) Yineng Zhang 2024-07-26 21:00:51 +10:00
• 40facad5f1 feat: support token ids in /v1/completions (#736) Toshiki Kataoka 2024-07-26 18:53:17 +09:00
• da504445dc fix /generate without sampling_params (#734) Toshiki Kataoka 2024-07-26 17:27:56 +09:00
• 252e0f7bbd fix: small bug for llama-405b fp16 (#733) Ying Sheng 2024-07-25 21:14:54 -07:00
• 7f6f2f0f09 Update readme (#731) Ying Sheng 2024-07-25 09:13:37 -07:00
• 7802df1e2b Update readme Ying Sheng 2024-07-25 08:14:36 -07:00
• 1a491d00cb Bump version to 0.2.0 (#730) Ying Sheng 2024-07-25 08:03:36 -07:00
• 8fbba3de3d Fix bugs (fp8 checkpoints, triton cache manager) (#729) Ying Sheng 2024-07-25 07:42:00 -07:00
• ae0f6130cb Revert "fix: fp8 config" (#728) Ying Sheng 2024-07-25 07:25:33 -07:00
• 6010589783 misc: update bug issue template (#727) Yineng Zhang 2024-07-25 20:52:37 +10:00
• 926ac01b64 fix: resolve the logo display issue on the PyPI page (#726) Yineng Zhang 2024-07-25 20:47:46 +10:00
• 25c881a005 chore: bump v0.1.25 (#725) Yineng Zhang 2024-07-25 20:04:35 +10:00
• 04ec6ba2ac Fix dockerfile and triton cache manager (#720) Liangsheng Yin 2024-07-25 03:04:21 -07:00
• d63f13c13b fix: fp8 config (#723) Ying Sheng 2024-07-25 02:01:56 -07:00
• fded67441d misc: update bulid instruction (#724) Yineng Zhang 2024-07-25 17:08:11 +10:00
• 6e45394051 chore: add close inactive issues workflow (#722) Yineng Zhang 2024-07-25 16:31:23 +10:00
• 97e0f7d250 docs: update comment (#721) Yineng Zhang 2024-07-25 10:51:18 +10:00
• d5146baec9 docs: update supported models (#719) Yineng Zhang 2024-07-25 09:34:01 +10:00
• 459abad261 Bump version to 0.1.24 (#718) Ying Sheng 2024-07-24 15:55:01 -07:00
• 30d8e130e7 Improve benchmark scripts (#717) Ying Sheng 2024-07-24 14:44:14 -07:00
• 08a3bd19cc docs: update doc (#716) Ying Sheng 2024-07-24 13:38:06 -07:00
• 321a963b01 misc: update doc (#715) Yineng Zhang 2024-07-25 06:05:46 +10:00
• e17deb27b5 fix: llama 3.1 405b fp8 (#714) Yineng Zhang 2024-07-25 02:37:41 +10:00
• 2d3ae4e125 docs: update doc (#713) Yineng Zhang 2024-07-25 00:03:17 +10:00
• 75f4ccb7dd docs: update README (#712) Yineng Zhang 2024-07-24 23:33:28 +10:00
• 83d2b30d75 format Ying Sheng 2024-07-24 10:53:07 +00:00
• 4367f4bb8d Fix prefill size (#711) Ying Sheng 2024-07-24 03:41:15 -07:00
• 00e4baa728 Update schedule_heuristic.py Lianmin Zheng 2024-07-24 01:22:30 -07:00
• 4cd64b8ee6 Auto adjust new ratio (#708) Liangsheng Yin 2024-07-23 22:06:02 -07:00
• 01d66ae2e8 Fix multi-node deadlock (#709) Lianmin Zheng 2024-07-23 21:53:36 -07:00
• a523a3c13a Reduce hardcoded logic of kernel usage (#707) Mingyi 2024-07-23 16:42:21 -07:00
• 9f94728f5a bump version to 0.1.23 (#706) Ying Sheng 2024-07-23 13:53:19 -07:00
• 444a02441a Update vllm version to support llama3.1 (#705) Ying Sheng 2024-07-23 13:49:34 -07:00
• fa7ccb3316 feat: add e2e latency (#704) zhyncs 2024-07-24 05:51:10 +10:00
• 268684439b Use min new token ratio at start (#701) Liangsheng Yin 2024-07-23 11:52:50 -07:00
• 824a77d04d Fix hf config loading (#702) Ke Bao 2024-07-24 02:39:08 +08:00
• cf99eab7d5 Fix flashinfer (#700) Ying Sheng 2024-07-23 01:27:01 -07:00
• 9fdea29d05 misc: fix typo (#698) zhyncs 2024-07-23 02:00:27 +10:00
• df7c4c19b4 Fix trt benchmark (#697) Ying Sheng 2024-07-22 06:32:41 -07:00
• c3f1aac811 Tune params (#696) Ying Sheng 2024-07-22 03:19:24 -07:00
• d198791fe8 misc: update output token logic (#695) zhyncs 2024-07-22 19:34:05 +10:00
• c07526e46c fix: update bench serving (#694) zhyncs 2024-07-22 18:23:33 +10:00
• 7b597475f2 docs: update README (#692) zhyncs 2024-07-22 03:41:20 +10:00
• 5303c1ed22 Support Mistral-Nemo (#691) Ke Bao 2024-07-22 01:36:53 +08:00
• 65bd13386b misc: recommend to use chat model for benchmark (#690) zhyncs 2024-07-22 00:13:33 +10:00
• eedc12e12e Support Deepseek MoE Model (#689) Liangsheng Yin 2024-07-21 03:09:29 -07:00
• 5a4ef2b5c8 update readme Lianmin Zheng 2024-07-21 02:58:57 -07:00
• 9dab947d56 docs: update README (#688) zhyncs 2024-07-21 18:32:58 +10:00
• 33ee97b0bf Allow disabling streaming in bench (#687) Lianmin Zheng 2024-07-21 01:12:34 -07:00
• 6a846bb1fd misc: update output file logic (#686) zhyncs 2024-07-21 18:07:30 +10:00
• 0fdb3127a1 feat: update bench serving (#685) zhyncs 2024-07-21 16:46:58 +10:00
• 5ad033a070 Fix StreamExecutor.fork() losing the current role start index. (#684) Max Shawabkeh 2024-07-20 23:32:11 -07:00
• 77e592e8e0 support non-streaming benchmark (#682) Lianmin Zheng 2024-07-20 18:36:42 -07:00
• caaad53b52 Support gpt-bigcode model class (#681) Liangsheng Yin 2024-07-20 18:34:37 -07:00
• 69d19188fc Decouple kv (#679) Liangsheng Yin 2024-07-20 14:16:45 -07:00
• 4b4a67f814 feat: support TRT LLM benchmark and multiple benchmarks (#670) zhyncs 2024-07-21 04:05:35 +10:00
• 0ac94c36cb Fallback when sampling failed (#678) Ke Bao 2024-07-21 01:44:54 +08:00
• 2b4c646277 Update version to 0.1.22 (#677) Ying Sheng 2024-07-20 03:39:50 -07:00
• f424e76d96 Fix illegal tokens during sampling (#676) Liangsheng Yin 2024-07-20 03:11:15 -07:00
• 490a1f39dd Fix cuda graph with flashinfer (#675) Lianmin Zheng 2024-07-20 02:43:55 -07:00
• 06487f126e refactor model loader: initial refactor (#664) Ying Sheng 2024-07-20 02:18:22 -07:00
• 39c57317e1 Revert "Temporary fix invalid sample results" (#673) Liangsheng Yin 2024-07-20 02:06:31 -07:00
• 9592a1f3bd Fix random dataset (#671) Lianmin Zheng 2024-07-20 01:57:43 -07:00
• 35759efa91 Support random dataset in bench_serving.py (#669) Lianmin Zheng 2024-07-20 01:06:43 -07:00
• 8f4b1559e7 Temporary fix invalid sample results (#668) Liangsheng Yin 2024-07-20 00:51:05 -07:00
• e3046ea3a8 Update OpenAI API (#667) Mingyi 2024-07-19 23:20:54 -07:00
• 49c5e0eca9 Add support for OpenAI API parallel sampling (#640) yichuan~ 2024-07-20 14:10:01 +08:00
• ec2150b294 Fix kill process util (#666) Ke Bao 2024-07-20 12:43:11 +08:00
• 7620cd37dd Fix jump forward when streaming (#665) Liangsheng Yin 2024-07-19 16:42:06 -07:00
• 50a53887be Update docs Ying Sheng 2024-07-19 11:40:06 -07:00
• 11c8efff73 Add benchmark instructions (#663) Ying Sheng 2024-07-19 11:12:23 -07:00
• e87c7fd501 Improve docs (#662) Ying Sheng 2024-07-19 10:58:03 -07:00
• 630479c3a6 feat: update check env (#661) zhyncs 2024-07-20 02:54:15 +10:00
• 51fda1439f Update Readme (#660) Ying Sheng 2024-07-19 09:54:01 -07:00
• dc4e4a6acc misc: update SGLang package description (#659) zhyncs 2024-07-20 02:27:39 +10:00
• 2d96da813e refactor model loader [unreachable code]: initial refactor (#655) Ying Sheng 2024-07-19 09:27:06 -07:00
• c126a6ccba feat: add benchmark serving (#657) zhyncs 2024-07-20 02:15:21 +10:00
• ac971ff633 perf: reduce ttft and itl with stream_interval 1 (#658) zhyncs 2024-07-20 02:14:22 +10:00
• e1792cca24 Remove cached triton launcher (#656) Lianmin Zheng 2024-07-18 23:28:40 -07:00
• 1b7adbb5a0 TokenizerManager.context_len should inherit from `server_args.conte… (#654) shrirajh 2024-07-19 14:25:29 +09:30
• a9ef49c12c Detokenize incrementally when streaming (#653) Liangsheng Yin 2024-07-18 17:57:40 -07:00
• 21ba3a88a1 Remove useless variables in infer_batch.py (#651) Ying Sheng 2024-07-18 05:31:44 -07:00
• 9c5cac2450 fix: resolve lint error (#650) zhyncs 2024-07-18 20:33:21 +10:00
• 5960a6e505 feat: add lint workflow (#648) zhyncs 2024-07-18 20:04:30 +10:00