Commit Graph

Select branches

Hide Pull Requests

cch/megamoe-rc-v0.1

Mono Color

• 929c7621af Fix: incorrect top_logprobs in chat completion (#2088) Alexander Waitz 2024-11-19 04:21:36 -08:00
• b7a065eae3 Use cuda event wait and synchronization instead of busy waiting (#2089) Lianmin Zheng 2024-11-19 00:21:46 -08:00
• b110453802 Simplify logits penalizer (#2086) Lianmin Zheng 2024-11-18 17:48:28 -08:00
• 3b44bbeecf Allow passing extra request body to bench_offline_throughput.py (#2085) Lianmin Zheng 2024-11-18 14:59:15 -08:00
• 80e2c4a8de Fix chunked prefill with output logprob (#2083) Lianmin Zheng 2024-11-18 13:16:28 -08:00
• 66318ffe96 Rename layer_idx to layer_id for consistency (#2078) Jani Monoses 2024-11-18 23:00:02 +02:00
• 766192610e feat: update torch 2.5.1 (#2069) Yineng Zhang 2024-11-18 21:29:13 +08:00
• 2a3992b6f1 support set role as 'tool' (#2075) yukavio 2024-11-18 17:06:59 +08:00
• 4af3f889fc Simplify flashinfer indices update for prefill (#2074) Lianmin Zheng 2024-11-18 00:02:36 -08:00
• df7fe4521a Crash the CI jobs on model import errors (#2072) Lianmin Zheng 2024-11-17 22:18:11 -08:00
• a7164b620f Tune the threshold for accuracy tests in CI (#2071) Lianmin Zheng 2024-11-17 21:51:00 -08:00
• 116685337e Fix cuda illegal memory access in overlap mode (#2070) Lianmin Zheng 2024-11-17 21:29:30 -08:00
• a9e90b4bce [Minor] Fix styles for overlap mode (#2068) Lianmin Zheng 2024-11-17 19:49:20 -08:00
• 8c280cee55 add phi-3 small support (#2062) Tanjiro 2024-11-17 18:47:43 -08:00
• 9c745d078e [Performance] Update xgrammar-related constrained decoding (#2056) DarkSharpness 2024-11-18 09:58:49 +09:00
• ebaa2f3199 Rename arguments --disable-nan-detection to --enable-nan-detection (#2066) Lianmin Zheng 2024-11-17 16:53:44 -08:00
• 62832bb272 Support cuda graph for DP attention (#2061) Ke Bao 2024-11-18 08:29:20 +08:00
• 11f881d173 Deprecate --disable-flashinfer and --disable-flashinfer-sampling (#2065) Lianmin Zheng 2024-11-17 16:20:58 -08:00
• 38625e2139 Remove monkey_patch_vllm_dummy_weight_loader (#2064) Lianmin Zheng 2024-11-17 15:48:12 -08:00
• c1f401fc58 Revert "chore: update torch v2.5.1" (#2063) Lianmin Zheng 2024-11-17 15:29:38 -08:00
• 3b878863f7 chore: update torch v2.5.1 (#1849) Yineng Zhang 2024-11-18 00:06:00 +08:00
• f719d9aebc Launch dp ranks in parallel (#2053) Lianmin Zheng 2024-11-16 17:13:36 -08:00
• edad373135 Fix illegal memory access in overlap mode & Use more fused triton kernels for building meta data (#2051) Lianmin Zheng 2024-11-16 16:14:23 -08:00
• 976bc302e5 Support DP MLA (#1970) Ke Bao 2024-11-16 17:01:43 +08:00
• 2f2e07439c Fix weight update for data parallelism (#2050) Lianmin Zheng 2024-11-16 00:30:39 -08:00
• 2ffe0a7363 Add get_amdgpu_memory_capacity() (#2049) HAI 2024-11-15 22:51:48 -08:00
• cf2489762b Add Tensor Parallel to torch_native_llama (#1876) Ke Wen 2024-11-15 21:26:00 -08:00
• e5c6715003 Fix core (MI300X) with --enable-overlap (#2048) HAI 2024-11-15 21:24:42 -08:00
• 023d0a73df fix small typos in docs (#2047) Xiaoyu Zhang 2024-11-16 03:09:10 +08:00
• 32c9a7ec11 Release v0.3.5.post2 (#2046) Lianmin Zheng 2024-11-15 06:54:00 -08:00
• b01df48cf2 [Fix] Adjust default chunked prefill size and cuda graph max bs according to GPU memory capacity (#2044) Lianmin Zheng 2024-11-15 06:21:57 -08:00
• c29b98e043 Fix json benchmark (#2043) Lianmin Zheng 2024-11-15 05:33:43 -08:00
• 954f4e6bd6 benchmark json schema (#2030) DarkSharpness 2024-11-15 22:06:19 +09:00
• 2558d6a675 Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042) Lianmin Zheng 2024-11-15 05:02:44 -08:00
• 29ebe3dff4 fix: align enable_overlap_scheduler naming between code and docs (#2038) ws 2024-11-15 19:39:10 +08:00
• f6dd648620 Offline LLM Engine Benchmark Throughput (#1968) zolinthecow 2024-11-14 21:59:33 -08:00
• ea53c63bad Expose no_stop_trim and skip_special_tokens in openai api (#2039) Lianmin Zheng 2024-11-14 19:09:21 -08:00
• a10d530943 Fix outlines version (#2036) Lianmin Zheng 2024-11-14 12:52:40 -08:00
• aae5434bdf Fix unit tests (#2034) Lianmin Zheng 2024-11-14 11:08:37 -08:00
• c3eac1b010 Fix torch.compile for MoE (#2033) Lianmin Zheng 2024-11-14 01:30:24 -08:00
• b275ce0043 Github runner instructions for AMD (#2031) HAI 2024-11-13 23:57:18 -08:00
• 13ce3e4b5d Add download_dir ServerArgs property (#2027) Patrick Yi 2024-11-14 02:26:56 -05:00
• df246e699d chore: open lto and optimization in release profile (#2028) Tzu Gwo 2024-11-14 15:02:39 +08:00
• fb9fb3518b set content to empty string (#2026) chottolabs 2024-11-13 20:06:02 -05:00
• c722d9bdc3 Fix dependency and error message for xgrammar (#2024) Lianmin Zheng 2024-11-13 14:04:25 -08:00
• 218ab3611d Do not let invalid grammar crash the server (#2023) Lianmin Zheng 2024-11-13 11:39:16 -08:00
• f407fcf9ef Release v0.3.5.post1 (#2022) Lianmin Zheng 2024-11-13 10:27:12 -08:00
• 54479d6f30 Fix grammar backend for tensor parallelism (#2020) Lianmin Zheng 2024-11-13 01:49:45 -08:00
• ba069a24d3 Fix grammar backend (#2018) Lianmin Zheng 2024-11-12 21:17:38 -08:00
• 125b1199c5 support parallel grammar preprocessing (#1996) DarkSharpness 2024-11-13 01:45:28 +09:00
• eff468dd5a fix test_embedding_models prompt length too long's bug (#2015) Xiaoyu Zhang 2024-11-12 23:21:16 +08:00
• a1bd719031 fix a bug in v1_embeeding_request (#2014) Xiaoyu Zhang 2024-11-12 16:49:45 +08:00
• 78c1d6445f Fix finish reason (#2013) Lianmin Zheng 2024-11-11 23:24:41 -08:00
• 027e65248f support echo=true and logprobs in openai api when logprobs=1 in lm-evaluation-harness (#1998) Xiaoyu Zhang 2024-11-12 15:21:20 +08:00
• b808a38365 Filter empty prompt in random bench serving (#2011) Ke Bao 2024-11-12 14:53:41 +08:00
• 602ebc661d remove sglang folder in rust (#2010) Byron Hsu 2024-11-11 20:45:52 -08:00
• 530ae1bdc8 Fix weight loading for tied word embedding when TP > 1 (#2009) Lianmin Zheng 2024-11-11 17:52:42 -08:00
• befc6beb86 Fix a typo in io_struct.py (#2008) Lianmin Zheng 2024-11-11 16:34:10 -08:00
• 59a5ba9be0 [Minor] Remove unused imports (#2006) Lianmin Zheng 2024-11-11 15:36:14 -08:00
• 86c37d010a fix sglang_router not found (#2005) Byron Hsu 2024-11-11 15:20:14 -08:00
• f18b9c7252 support internlm2-reward (#1994) RangiLyu 2024-11-12 07:09:58 +08:00
• 3e33574374 run rust test on ubuntu instead of 1-gpu-runner (#2003) Byron Hsu 2024-11-11 14:46:08 -08:00
• 0d94f1dd03 Bump router to 0.0.3 (#2004) Byron Hsu 2024-11-11 14:42:22 -08:00
• e728258d34 release router from py38 to py312 (#2002) Byron Hsu 2024-11-11 14:30:25 -08:00
• 239eafbd2e Fix rust unit test and pypi token (#2001) Byron Hsu 2024-11-11 14:18:21 -08:00
• 9d427265fd Add Engine::encode example (#2000) James Xu 2024-11-11 16:43:35 -05:00
• 00ffde206f setup router python binding ci (#1999) Byron Hsu 2024-11-11 12:19:32 -08:00
• ddeb9d42de Add engine encode (#1995) James Xu 2024-11-11 14:48:17 -05:00
• aaf0a3156e docs: add slides link in README (#1997) Yineng Zhang 2024-11-11 21:03:16 +08:00
• f9633fa9b9 [rust] cache-aware DP - approx tree (#1934) Byron Hsu 2024-11-10 21:57:32 -08:00
• 087ab83223 [Performance, Triton] Optimize over mask compute to tl.load in fused_moe_kernel (#1980) HAI 2024-11-10 18:54:43 -08:00
• 8169c6f4ef Add gen-shared-prefix dataset in bench_serving (#1990) Byron Hsu 2024-11-10 16:39:56 -08:00
• 3d043319aa [CI] Balance unit tests (#1988) Lianmin Zheng 2024-11-10 11:45:01 -08:00
• a8aad9357d qwen2vl fix bug for #1971 #1897 (#1984) yizhang2077 2024-11-11 00:10:45 +08:00
• 47ffe7af81 docs: add shm size for docker run (#1986) Yineng Zhang 2024-11-10 22:14:48 +08:00
• b3523af8eb fix: update pyzmq version (#1983) Yineng Zhang 2024-11-10 21:33:23 +08:00
• 1929c06762 Simplify prometheus metrics (#1981) Lianmin Zheng 2024-11-10 04:39:32 -08:00
• ed53ac84b4 Specify zmq Version Requirement (#1982) Huanzhi (Hans) Mao 2024-11-10 01:32:07 -08:00
• 520f0094e4 [CI] balance unit tests (#1977) Lianmin Zheng 2024-11-09 16:46:14 -08:00
• 9c939a3d8b Clean up metrics code (#1972) Lianmin Zheng 2024-11-09 15:43:20 -08:00
• 549e8b8366 [Minor] Fix a typo in test_torchao.py (#1976) Lianmin Zheng 2024-11-09 15:07:27 -08:00
• a1f32867ca Update pr-test-rust.yml to add a "finish" step (#1975) Lianmin Zheng 2024-11-09 13:53:35 -08:00
• 760552e068 Update README.md (#1974) Lianmin Zheng 2024-11-09 11:32:13 -08:00
• d9aada9db1 Introducing SGLang Guru on Gurubase.io (#1745) Kursat Aktas 2024-11-09 22:29:26 +03:00
• f11eb90fe4 Initialize model_worker_batch variable (#1973) Enrique Shockwave 2024-11-09 19:28:02 +00:00
• 95a4ed129a Fix metrics (#1963) Yudi Xue 2024-11-08 23:21:11 -08:00
• d1150e9a00 Updated Instructions on Profiling SGLang Infer System with AMD GPUs (#1966) leishaoSC 2024-11-08 23:19:03 -08:00
• e3126e3c5f Update README.md's Slack invitation link (#1962) Chayenne 2024-11-08 11:46:25 -08:00
• a509552087 [minor] Improve code style and compatibility (#1961) Lianmin Zheng 2024-11-08 02:19:41 -08:00
• 7ef0084b0d Add sentence_transformers to CI dependency (#1958) Lianmin Zheng 2024-11-08 01:21:29 -08:00
• f9a377f650 [Release, ROCm] release ROCm docker build for AMD MI GPUs (#1957) HAI 2024-11-08 00:14:15 -08:00
• 4ade15dd32 Adjust reward model's score module and pooler module order for reducing computation (#1956) aqweteddy 2024-11-08 16:10:54 +08:00
• 8dc84da084 Remove the useless to_srt_kwargs (#1955) Lianmin Zheng 2024-11-07 23:15:08 -08:00
• f16eb15d0d Gemma2 reward model support (#1954) aqweteddy 2024-11-08 14:42:27 +08:00
• 5bc2508b80 Monitoring documentation (#1933) Yudi Xue 2024-11-07 22:14:16 -08:00
• a71a44f203 Update setup_github_runner.md (#1952) Lianmin Zheng 2024-11-07 19:20:47 -08:00
• 691808d587 Add a timeout for execute-notebook.yml (#1951) Lianmin Zheng 2024-11-07 18:28:29 -08:00
• d32fba2a4d [ENV, ROCm] update environment settings (#1939) HAI 2024-11-07 18:24:36 -08:00
• 67c424cce3 [Performance, Triton Kernel Args] extend_attention, optimize kern args to _fwd_kernel (#1941) HAI 2024-11-07 18:24:02 -08:00
• 1ae270c5d0 [Doc] fix docs (#1949) Lianmin Zheng 2024-11-07 18:20:41 -08:00