Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• f4d9953d9d misc: add triton in check_env PACKAGE_LIST (#914) Yineng Zhang 2024-08-04 21:20:59 +08:00
• 4f00525057 fix: use e2e and unit test only for original repo or pr (#912) Yineng Zhang 2024-08-04 14:34:50 +08:00
• 995af5a54b Improve the structure of CI (#911) Ying Sheng 2024-08-03 23:09:21 -07:00
• 539856455d latency test enhancement - part 1 (#909) min-xu-et 2024-08-03 22:44:58 -07:00
• 70cc0749ce Add model accuracy test - step 1 (#866) Ying Sheng 2024-08-03 18:20:50 -07:00
• 7dd8a7e6d9 fixed an error handling in bench_latency.py (#904) min-xu-et 2024-08-03 17:42:17 -07:00
• 947402c829 Reorder CI unit tests. (#908) Liangsheng Yin 2024-08-03 16:18:50 -07:00
• 8c5382e62c Update README.md Ying Sheng 2024-08-03 12:58:41 -07:00
• 001b0bdd08 Update the base image of the docker (#900) Ying Sheng 2024-08-02 21:54:57 -07:00
• b906c01592 Bump version to 0.2.9.post1 (#899) Ying Sheng 2024-08-02 12:08:00 -07:00
• 9319cd139c [minor] fixed code formatting doc (#896) min-xu-et 2024-08-02 09:39:28 -07:00
• 046c2b339e chore: add multipart dep for fastapi (#895) Yineng Zhang 2024-08-02 22:50:19 +08:00
• 6b8f66efe1 misc: update cuda graph capture exception log (#894) Yineng Zhang 2024-08-02 22:40:52 +08:00
• 7937a886b2 docs: update setup runner (#884) Yineng Zhang 2024-08-02 19:03:53 +08:00
• 2e218b9e04 fix: set env in runner (#891) Yineng Zhang 2024-08-02 18:48:56 +08:00
• 30a9b2ef20 Bump version to v0.2.9 (#890) Ying Sheng 2024-08-02 01:45:48 -07:00
• 3cadecf0c4 Increase openai client limit (#886) Ying Sheng 2024-08-02 00:47:23 -07:00
• e90e3a50d4 Add benchmark: HumanEval (#889) Ying Sheng 2024-08-02 00:46:41 -07:00
• fbd6b94d69 Fix the double BOS problem in the HF chat template (#888) Ying Sheng 2024-08-02 00:30:50 -07:00
• 4c8093c8db Update workflow name (#883) Ying Sheng 2024-08-01 21:29:46 -07:00
• ae7ee01a8e Add accuracy test to CI: MMLU (#882) Ying Sheng 2024-08-01 21:20:17 -07:00
• 76e59088d8 Add more unit tests to CI (#880) Ying Sheng 2024-08-01 18:14:33 -07:00
• 12ce3befb6 Update runner docs (#879) Liangsheng Yin 2024-08-01 17:37:47 -07:00
• 4013a4e1b0 Implement served_model_name to customize model id when use local mode… (#749) 任嘉 2024-08-02 08:13:51 +08:00
• 60340a3643 Improve the coverage of the openai api server test (#878) Ying Sheng 2024-08-01 16:01:30 -07:00
• 70c78cfb03 Update runner docs (#876) Liangsheng Yin 2024-08-01 15:32:33 -07:00
• 72b6ea88b4 Make scripts under /test/srt as unit tests (#875) Ying Sheng 2024-08-01 14:34:55 -07:00
• e4d3333c6c bump to 0.2.8 (#877) Ying Sheng 2024-08-01 14:18:26 -07:00
• 6f221d4ca0 Fix unit tests for the frontend language part (#872) Ying Sheng 2024-08-01 12:39:12 -07:00
• aba6f51f88 misc: update unit test config (#873) Yineng Zhang 2024-08-02 03:27:05 +08:00
• 7f6c690b67 misc: use pip cache purge and add unit test ci (#871) Yineng Zhang 2024-08-02 03:12:20 +08:00
• 40e6f5131a Fix openai CI tests (#870) Ying Sheng 2024-08-01 09:39:09 -07:00
• 4075677621 Add OpenAI backend to the CI test (#869) Ying Sheng 2024-08-01 09:25:24 -07:00
• 9e8d2c7f74 misc: add cancel previous at e2e (#864) Yineng Zhang 2024-08-01 16:26:54 +08:00
• c9bff5fcc8 misc: disable auto release (#862) Yineng Zhang 2024-08-01 15:46:51 +08:00
• b04444ac01 Rename github workflows (#861) Ying Sheng 2024-08-01 00:39:55 -07:00
• 3d617a21ba misc: update e2e test paths config (#860) Yineng Zhang 2024-08-01 15:38:24 +08:00
• c020f9ceda Support chunked prefill when radix cache is disabled (#811) Liangsheng Yin 2024-08-01 00:29:01 -07:00
• ca600e8cd6 Add support for logprobs in OpenAI chat API (#852) yichuan~ 2024-08-01 15:08:21 +08:00
• 0c0c81372e Fix #857 (#858) Kai Fronsdal 2024-08-01 00:05:39 -07:00
• 90286d8576 Add troubleshooting doc (#856) Ying Sheng 2024-08-01 00:05:26 -07:00
• 5e7dd984fe Fix llama for classification (#855) Ying Sheng 2024-07-31 15:48:31 -07:00
• bc3eaac2b8 chore: update flashinfer to v0.1.3 (#850) Yineng Zhang 2024-08-01 02:37:05 +08:00
• a78d98de19 misc: update e2e test paths config (#848) Yineng Zhang 2024-07-31 16:37:29 +08:00
• 7d5ed7c6ee docs: update README.md (#843) Ikko Eltociear Ashimine 2024-07-31 11:48:18 +09:00
• a6c7ebbbcb Add req slots leaking check (#842) Liangsheng Yin 2024-07-30 18:29:01 -07:00
• bb0501c0d9 Fix List input bug (#838) yichuan~ 2024-07-31 04:40:51 +08:00
• 6b0f2e9088 Add --max-total-tokens (#840) Liangsheng Yin 2024-07-30 13:33:55 -07:00
• 1edd4e07d6 chore: bump v0.2.7 (#830) Yineng Zhang 2024-07-30 20:41:10 +10:00
• 62c673c46f docs: add set up runner (#829) Yineng Zhang 2024-07-30 19:43:40 +10:00
• 377c5dc9a9 misc: enable e2e test when push (#828) Yineng Zhang 2024-07-30 19:26:23 +10:00
• f52eda35ea misc: update e2e test benchmark config (#825) Yineng Zhang 2024-07-30 19:19:23 +10:00
• b579ecf028 Add awq_marlin (#826) Ying Sheng 2024-07-30 02:04:51 -07:00
• e7487b08bc Adjust default mem fraction to avoid OOM (#823) Ying Sheng 2024-07-30 01:58:31 -07:00
• ae5c0fc442 Support disable_ignore_eos in bench_serving.py (#824) Ying Sheng 2024-07-30 01:42:07 -07:00
• a30d5d75bf feat: add pr e2e test (#822) Yineng Zhang 2024-07-30 18:31:26 +10:00
• 17af39c5dc feat: add runner (#821) Yineng Zhang 2024-07-30 17:32:13 +10:00
• daf593a385 Fix streaming bug (#820) ObjectNotFound 2024-07-30 15:32:07 +08:00
• bece265f5a docs: update README (#819) Yineng Zhang 2024-07-30 16:17:50 +10:00
• cdcbde5fc3 Code structure refactor (#807) Liangsheng Yin 2024-07-29 23:04:48 -07:00
• 21e22b9e96 Fix LiteLLM kwargs (#817) Enrique Shockwave 2024-07-30 06:38:02 +01:00
• a50c8a14b3 fix: use v0.2.5 for benchmark (#814) Yineng Zhang 2024-07-30 12:40:35 +10:00
• db6089e6f3 Revert "Organize public APIs" (#815) Ying Sheng 2024-07-29 19:40:28 -07:00
• 3520f75fb1 Remove inf value for chunked prefill size (#812) Liangsheng Yin 2024-07-29 18:34:25 -07:00
• c8e9fed87a Organize public APIs (#809) Liangsheng Yin 2024-07-29 15:34:16 -07:00
• 084fa54d37 Add support for OpenAI API : offline batch(file) processing (#699) yichuan~ 2024-07-30 04:07:18 +08:00
• eba458bd19 Revert "Revert "fix: update flashinfer to 0.1.2 to fix sampling for cu118"" (#806) Ying Sheng 2024-07-29 12:20:42 -07:00
• 3d1cb0af83 feat: add chat template for internlm2-chat (#802) Yineng Zhang 2024-07-30 05:18:03 +10:00
• 7d352b4fdd Revert "fix: update flashinfer to 0.1.2 to fix sampling for cu118" (#805) Ying Sheng 2024-07-29 11:39:12 -07:00
• 87064015d9 fix: update flashinfer to 0.1.2 to fix sampling for cu118 (#803) Yineng Zhang 2024-07-30 04:00:52 +10:00
• 7cd4f244a4 Chunked prefill (#800) Liangsheng Yin 2024-07-29 03:32:58 -07:00
• 98111fbe3e Revert "Chunked prefill support" (#799) Ying Sheng 2024-07-29 02:38:31 -07:00
• 2ec39ab712 Chunked prefill support (#797) Liangsheng Yin 2024-07-29 02:21:50 -07:00
• 8f6274c82b Add role documentation, add system begin & end tokens (#793) ObjectNotFound 2024-07-29 14:02:49 +08:00
• 325a06c2de Fix logging (#796) Ying Sheng 2024-07-28 23:01:45 -07:00
• 79f816292e Fix lazy import location (#795) Ying Sheng 2024-07-28 22:09:50 -07:00
• b688fd858d Lazy-import third-party backends (#794) Eric Yoon 2024-07-29 13:57:41 +09:00
• 5bd899243b Update README.md (#792) Ying Sheng 2024-07-28 21:57:23 -07:00
• 8d908a937c Fix echo + lobprob for OpenAI API when the prompt is a list (#791) Ying Sheng 2024-07-28 17:09:16 -07:00
• dd7e8b9421 chore: add copyright for srt (#790) Yineng Zhang 2024-07-28 23:07:12 +10:00
• 1f013d64eb docs: make badges center (#789) Yineng Zhang 2024-07-28 22:27:52 +10:00
• 628e1fa760 docs: update README (#788) Yineng Zhang 2024-07-28 22:24:27 +10:00
• c71880f896 Vectorize logprobs computation (#787) Ying Sheng 2024-07-28 05:22:14 -07:00
• bcb6611a46 Update README.md Ying Sheng 2024-07-28 01:00:06 -07:00
• fa2aa0db0a docs: update index (#786) Yineng Zhang 2024-07-28 17:22:00 +10:00
• 6a387a69cc fix: exclude logo png in gitignore (#785) Yineng Zhang 2024-07-28 17:08:16 +10:00
• 27f5ce0a6c fix: init readthedocs support (#784) Yineng Zhang 2024-07-28 16:55:54 +10:00
• 948625799e docs: init readthedocs support (#783) Yineng Zhang 2024-07-28 16:50:31 +10:00
• 68e5262699 fix: replace pillow with PIL in PACKAGE_LIST (#781) Yineng Zhang 2024-07-28 14:06:24 +10:00
• bc1154c399 Bump version to 0.2.6 (#779) Lianmin Zheng 2024-07-27 20:29:33 -07:00
• 752e643007 Allow disabling flashinfer sampling kernel (#778) Lianmin Zheng 2024-07-27 20:18:56 -07:00
• 30db99b3d9 Rename prefill_token_logprobs -> input_token_logprobs; decode_token_logprobs -> output_token_logprobs (#776) Lianmin Zheng 2024-07-27 19:50:34 -07:00
• 0a409bd438 Fix return_log_probs with cuda graph (#775) Lianmin Zheng 2024-07-27 19:15:09 -07:00
• e4db4e5ba5 minor refactor: move check server args to server_args.py (#774) Mingyi 2024-07-27 19:03:40 -07:00
• bbc07c4197 Move sampling logits to float32 (#773) Lianmin Zheng 2024-07-27 17:30:12 -07:00
• a036d41980 Fix max new tokens (#772) Lianmin Zheng 2024-07-27 17:22:18 -07:00
• f95e661757 Fix max_tokens for OpenAI chat completion API (#766) Lianmin Zheng 2024-07-27 15:44:27 -07:00
• de854fb5c5 feat: add fake tag (#770) Yineng Zhang 2024-07-28 02:22:22 +10:00
• f64b2a9bc0 Add slack invitation link. Lianmin Zheng 2024-07-27 06:29:15 -07:00
• 9f95dcc64f Update readme (#769) Ying Sheng 2024-07-27 06:12:16 -07:00