Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• c77c1e05ba fix black in pre-commit (#1940) Chayenne 2024-11-07 15:42:47 -08:00
• dca87ec348 [Docs] fix 404 - Contributor Guide (#1942) HAI 2024-11-07 00:50:45 -08:00
• 4b1d7a2583 Add Rust Router Python Binding (#1891) Austin Liu 2024-11-07 10:08:30 +08:00
• a5e0defb5a minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces (#1926) Xuehai Pan 2024-11-06 21:46:04 +08:00
• 96766101b4 [rust] refactor server and router (#1922) Byron Hsu 2024-11-06 00:02:02 -08:00
• a146d9990e support prometheus metrics (#1853) Lzhang-hub 2024-11-06 12:42:53 +08:00
• f5113e50ae [Doc] improve relative links and structure (#1924) Lianmin Zheng 2024-11-05 01:12:10 -08:00
• 02755768d3 Change judge to classify & Modify make file (#1920) Chayenne 2024-11-04 23:53:44 -08:00
• 463d56bf44 Update CODEOWNERS (#1916) Byron Hsu 2024-11-04 17:13:41 -08:00
• 530ff541cf [router] Impl radix tree and set up CI (#1893) Byron Hsu 2024-11-04 10:56:52 -08:00
• 3cd2809277 [Docs, ROCm] update install to cover ROCm with MI GPUs (#1915) HAI 2024-11-04 01:40:57 -08:00
• 704f8e8ed1 Add Reward API Docs etc (#1910) Chayenne 2024-11-03 22:33:03 -08:00
• 1853c3523b Fix regex docs (#1909) Lianmin Zheng 2024-11-03 14:18:16 -08:00
• 65859754f1 Release v0.3.5 (#1908) Lianmin Zheng 2024-11-03 13:48:11 -08:00
• 2ce32db6fb Let reward model take text inputs instead of message lists (#1907) Lianmin Zheng 2024-11-03 13:27:12 -08:00
• 793b79dbe9 feat: support truss endpoint for benchmark serving (#1906) Yineng Zhang 2024-11-03 12:56:10 -08:00
• 1363b51983 Escape backwards slash (#1902) Iñaki Arango 2024-11-03 12:27:11 -08:00
• 0abbf289a8 Unify the model type checking (#1905) Lianmin Zheng 2024-11-03 12:25:39 -08:00
• c17c578108 Simplify tokenizer manager (#1904) Lianmin Zheng 2024-11-03 08:38:26 -08:00
• 916b3cdddc Allow passing dtype and max_new_tokens to HF reference script (#1903) Jani Monoses 2024-11-03 18:24:37 +02:00
• 838dcda162 Simplify tokenizer manager (#1899) Lianmin Zheng 2024-11-03 03:52:38 -08:00
• efbc116a0f Do not use longest prefix matching when #queue-req is large (#1896) Lianmin Zheng 2024-11-03 01:45:20 -07:00
• 6aed0445ed turn off log (#1895) Chayenne 2024-11-03 00:19:12 -07:00
• 908dd7f9aa Add engine api (#1894) Chayenne 2024-11-02 22:03:38 -07:00
• f4cd804073 Fix ci and link error (#1892) Chayenne 2024-11-02 19:08:49 -07:00
• be7986e005 Fix docs (#1890) Lianmin Zheng 2024-11-02 13:26:32 -07:00
• 5a5f18432f Fix docs ci (#1888) Chayenne 2024-11-02 11:57:22 -07:00
• 7b394e5f2b Fix docs (#1889) Lianmin Zheng 2024-11-02 11:46:00 -07:00
• 3b60558dd7 Native api (#1886) Chayenne 2024-11-02 01:02:17 -07:00
• 5a9a4f41c6 Update index.rst (#1885) Lianmin Zheng 2024-11-02 00:20:33 -07:00
• 72e979bfb5 add native api docs (#1883) Chayenne 2024-11-02 00:17:30 -07:00
• 146f613405 Fix incorrect context length for llama3.2-11b (#1873) Ran Chen 2024-11-02 00:04:50 -07:00
• 660ecb731f Fix doc links (#1882) Lianmin Zheng 2024-11-01 20:42:30 -07:00
• 2565cb0f40 Update docs and workflow (#1881) Lianmin Zheng 2024-11-01 20:29:41 -07:00
• 066e8a4ef0 Update docs title (#1879) Lianmin Zheng 2024-11-01 20:00:41 -07:00
• 2134f0898c Fix links in the docs (#1878) Lianmin Zheng 2024-11-01 18:21:14 -07:00
• a54f278d44 Add a FAQ documentation (#1877) Lianmin Zheng 2024-11-01 18:16:29 -07:00
• d1b31b0684 Improve docs and fix the broken links (#1875) Lianmin Zheng 2024-11-01 17:47:44 -07:00
• d59a47828c [3rdparty, document] Updated Documentation that covers performance tuning techniques for AMD Instinct GPUs. (#1871) jacky.cheng 2024-11-02 03:12:59 +08:00
• 104bf2609b minor: update nightly eval (#1867) Yineng Zhang 2024-11-01 21:38:29 +08:00
• 3bf3d011ed Add vlm document (#1866) Chayenne 2024-11-01 00:51:15 -07:00
• d86a2d6562 minor: add human eval (#1754) Yineng Zhang 2024-11-01 14:29:20 +08:00
• 16eb33ffe2 Update vocab embedding deps and add TP switch (#1856) Ke Bao 2024-11-01 11:13:07 +08:00
• 61cf00e112 change file tree (#1859) Chayenne 2024-10-31 20:10:16 -07:00
• b9fd178f1b Fix retraction + overlap (#1860) Liangsheng Yin 2024-10-31 18:27:42 -07:00
• d8e9d61f86 [Build, ROCm] Dockerfile.rocm for Instinct GPUs, with package updates (#1861) HAI 2024-10-31 16:38:16 -07:00
• a2e0424abf Fix memory leak for chunked prefill 2 (#1858) Lianmin Zheng 2024-10-31 14:51:51 -07:00
• 8ce202a493 delete unused character (#1855) geeker-smallwhite 2024-10-31 19:33:55 +08:00
• d913d52c9a Fix warnings in doc build (#1852) Lianmin Zheng 2024-10-30 22:28:00 -07:00
• 0ab7bcaf66 Simplify documentation in README.md (#1851) Lianmin Zheng 2024-10-30 21:57:49 -07:00
• 438526a814 Refactor tokenizer manager (#1846) Byron Hsu 2024-10-30 21:32:18 -07:00
• f7102fbd2b Fix mixed chunked prefill (#1850) Lianmin Zheng 2024-10-30 21:20:41 -07:00
• a7a0a6886b Make decode log interval configurable (#1847) Byron Hsu 2024-10-30 19:59:20 -07:00
• 2d4ce1b792 [Performance, Triton Kernel Args] _decode_grouped_softmax_reducev_fwd… (#1845) HAI 2024-10-30 17:33:36 -07:00
• 4ba815b84e Fix suggest edit (#1842) Chayenne 2024-10-30 12:28:12 -07:00
• 5f65e2b830 [Performance, Hardware] MoE weights padding to AMD MI300x GPUs (#1836) HAI 2024-10-30 12:17:32 -07:00
• 4e2af03cfa [Production] Drain requests before exit when receive SIGTERM (#1838) Ying Sheng 2024-10-30 10:22:56 -07:00
• 3184aa95a7 Update README.md (#1840) Lianmin Zheng 2024-10-30 03:16:43 -07:00
• b548801ddb Update docs (#1839) Lianmin Zheng 2024-10-30 02:49:08 -07:00
• 539df95d2c Imporve openai api documents (#1827) Chayenne 2024-10-30 00:39:41 -07:00
• 5e00ddebc0 Add new model: Gpt2 (#1833) DanielC12321 2024-10-29 19:52:33 -05:00
• 54dd3ea122 [FP8 KV Cache, Mixtral] Avoid KeyError at loading pre-quantized FP8 m… (#1835) HAI 2024-10-29 13:58:03 -07:00
• d04899d7ca stop_str of qwen2-vl template should be a tuple not a str (#1834) yizhang2077 2024-10-30 04:30:41 +08:00
• 5010e0d2ca [3rdparty, document] Add 3rdparty/amd, with profiling and tuning instructions to be added (#1822) HAI 2024-10-29 10:51:02 -07:00
• 5e6c32657e Support setting use_thread in the run_program for easier debugging. (#1823) Yanyi Liu 2024-10-29 14:51:47 +08:00
• 680cad2023 fix get_memory_pool_size deadlock for DP (#1830) Byron Hsu 2024-10-28 23:07:14 -07:00
• 0a24eb850a Fix update_weights deadlock for DP (#1825) Byron Hsu 2024-10-28 12:02:23 -07:00
• 3839be2913 [Router] Add a rust-based router (#1790) Byron Hsu 2024-10-28 09:49:48 -07:00
• 6e13b650a9 Fix docs deploy ci (#1821) Chayenne 2024-10-27 21:03:41 -07:00
• 6fcd6d7d6d Support token ids in engine.generate (#1820) Byron Hsu 2024-10-27 14:02:34 -07:00
• c77762d57f Fix Triton decode kernel & ut (#1819) Ke Bao 2024-10-28 01:54:38 +08:00
• 51c81e339b Add openAI compatible API (#1810) Chayenne 2024-10-27 10:51:42 -07:00
• eaade87a42 Fix unit tests (#1817) Lianmin Zheng 2024-10-27 03:04:54 -07:00
• 86fc0d79d0 Add a watch dog thread (#1816) Lianmin Zheng 2024-10-27 02:00:50 -07:00
• 1be853ee69 Update hyperparameter_tuning.md (#1813) Lianmin Zheng 2024-10-26 21:08:44 -07:00
• 86e0dde555 Improve the user control of new_token_ratio (#1811) Lianmin Zheng 2024-10-26 16:39:41 -07:00
• 2b80978859 Provide an argument to set the maximum batch size for cuda graph (#1809) Lianmin Zheng 2024-10-26 15:09:33 -07:00
• 9d6fb08457 Fix docs ci (#1808) Chayenne 2024-10-26 11:23:51 -07:00
• ced362f7c6 Simplify our docs with complicated functions into utils (#1807) Chayenne 2024-10-26 10:44:11 -07:00
• 9084a86445 Update links (#1805) Lianmin Zheng 2024-10-26 04:46:01 -07:00
• 6aa94b967c Update ci workflows (#1804) Lianmin Zheng 2024-10-26 04:32:36 -07:00
• c26507484f fix int conversion for SGLANG_CPU_COUNT (#1803) Byron Hsu 2024-10-26 00:09:44 -07:00
• 07bf2e846a Allow consecutive ports when launching multiple sglang servers. (#1802) Liangsheng Yin 2024-10-25 23:43:24 -07:00
• a628dd8e31 Set ZMQ buffer size heuristic (#1801) Liangsheng Yin 2024-10-25 23:15:56 -07:00
• 1e8903414a Fix possible ZMQ hanging (#1800) Liangsheng Yin 2024-10-25 23:07:07 -07:00
• 715b16c140 Add support for ipynb (#1786) Chayenne 2024-10-25 20:48:35 -07:00
• 9ce8e1a93c move max_position_embeddings to the last (#1799) Hui Liu 2024-10-25 19:30:50 -07:00
• fb99aaa527 [Fix] Fix --skip-tokenizer-init (#1798) Lianmin Zheng 2024-10-25 18:51:59 -07:00
• b77a02cdfd [Performance] Support both xgrammar and outlines for constrained decoding (#1752) DarkSharpness 2024-10-26 06:47:02 +09:00
• 30643fed7f Release v0.3.4.post2 (#1796) Lianmin Zheng 2024-10-25 11:07:19 -07:00
• e646c5901e Fix logprob in the overlapped mode (#1795) Lianmin Zheng 2024-10-25 11:06:57 -07:00
• c555ce2ca2 Revert "Fix memory leak when doing chunked prefill" (#1797) Lianmin Zheng 2024-10-25 10:24:44 -07:00
• 40900baea7 [Fix] Fix the log parsing in chunked prefill uni tests (#1794) Lianmin Zheng 2024-10-25 08:31:08 -07:00
• a2f5e7555f Fix memory leak when doing chunked prefill (#1787) Liangsheng Yin 2024-10-25 08:01:17 -07:00
• 2148914e1b Fix log parsing in the chunked prefill unit tests (#1793) Lianmin Zheng 2024-10-25 08:00:55 -07:00
• def55bc876 Qwen2vl support cuda graph and disable radix cache (#1780) yizhang2077 2024-10-25 22:45:17 +08:00
• 86a2c473b7 [Fix] Fix seq_lens_sum for cuda graph runner in padded cases (#1789) Lianmin Zheng 2024-10-24 21:26:05 -07:00
• 1701b0db31 Enhance the test case for chunked prefill (#1785) Lianmin Zheng 2024-10-24 21:23:09 -07:00
• 384d85ba35 Re-introduce get_cuda_graph_seq_len_fill_value (#1783) Lianmin Zheng 2024-10-24 13:30:11 -07:00
• 605972195b check user-specified model_max_len with hf derived max_model_len (#1778) Xiaoyu Zhang 2024-10-25 03:40:36 +08:00