 Lianmin Zheng
|
94e167ea5a
|
Fix the default chunked prefill size (#2268)
|
2024-11-29 16:03:32 -08:00 |
|
|
|
262e370f78
|
[benchmark] Add fused_moe_triton benchmark and tuning tools (#2225)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
|
2024-11-29 13:36:45 -08:00 |
|
|
Yineng Zhang
|
419a57e771
|
minor: add sgl-kernel dir (#2261)
|
2024-11-30 02:27:35 +08:00 |
|
|
Yineng Zhang
|
fae4e5e99a
|
chore: bump v0.3.6.post3 (#2259)
|
2024-11-30 01:41:16 +08:00 |
|
|
Lianmin Zheng
|
afe1e46586
|
[Minor] fix the style for multimodal models (#2257)
|
2024-11-29 04:24:20 -08:00 |
|
|
Lianmin Zheng
|
f50a6cf443
|
Fix hash collision for multi modal models (#2256)
|
2024-11-29 03:15:58 -08:00 |
|
|
Lianmin Zheng
|
fe97a2d40f
|
Simplify tokenizer manager (#2254)
|
2024-11-29 02:18:51 -08:00 |
|
|
|
8b48496aaf
|
Revert "Revert "Add simple CPU offloading support"" (#2253)
Co-authored-by: Jani Monoses <jani.monoses@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
|
2024-11-28 23:58:54 -08:00 |
|
|
Ying Sheng
|
4057ea82c9
|
Revert "Add simple CPU offloading support" (#2252)
We'll re-add the commit to correctly ack Kaichao's authorship
|
2024-11-28 23:36:55 -08:00 |
|
|
Lianmin Zheng
|
4f2ee48ed1
|
Update backend.md (#2251)
|
2024-11-28 23:18:07 -08:00 |
|
|
Lianmin Zheng
|
71ff2728a1
|
Update backend.md (#2250)
|
2024-11-28 23:14:36 -08:00 |
|
|
Ying Sheng
|
b7038fec9b
|
[fix] Fix prefix caching for multi-image/video (#2239)
|
2024-11-28 12:08:13 -08:00 |
|
|
Enrique Shockwave
|
65fdb28929
|
fix missing launch server import (#2242)
|
2024-11-28 21:24:47 +08:00 |
|
|
Lianmin Zheng
|
b2ccf36d4d
|
Fix memory leak during abort (#2238)
|
2024-11-28 02:22:15 -08:00 |
|
|
Lianmin Zheng
|
d4fc1a70e3
|
Crash the server correctly during error (#2231)
|
2024-11-28 00:22:39 -08:00 |
|
|
Jani Monoses
|
db674e3d24
|
Add OLMo2 model. (#2233)
|
2024-11-28 00:15:20 -08:00 |
|
|
Lianmin Zheng
|
fb915bd1a2
|
Disable overlap scheduler for multimodal models (#2235)
|
2024-11-27 23:44:33 -08:00 |
|
|
Lianmin Zheng
|
09798b36cd
|
Fix chunked prefill size for bench_offline_throughput (#2234)
|
2024-11-27 23:37:20 -08:00 |
|
|
HAI
|
b79fffdcb5
|
Update Install Method 2. From source (#2232)
|
2024-11-27 22:46:55 -08:00 |
|
|
HAI
|
cd51758fad
|
Rename tuned MI300X config files for fused_moe_triton (#2228)
|
2024-11-27 21:18:51 -08:00 |
|
|
bjmsongandroot
|
91e5dbf554
|
add profile in offline benchmark & update doc (#2123)
Co-authored-by: root <bjmsong@126.com>
|
2024-11-27 14:57:13 -08:00 |
|
|
Lianmin Zheng
|
dd5eba4c88
|
Remove fused_moe_grok (#2223)
|
2024-11-27 14:28:55 -08:00 |
|
|
Baoyuan Qi
|
a4fd2f9b46
|
fix typo prompts (#2224)
|
2024-11-27 12:07:00 -08:00 |
|
|
Byron Hsu
|
92d1253e58
|
Bump sglang-router to 0.0.10 for env name change (#2226)
|
2024-11-27 11:23:32 -08:00 |
|
|
|
a9ca297d76
|
[3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs (#2191)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
|
2024-11-28 02:23:10 +08:00 |
|
|
Lianmin Zheng
|
2a02185c5f
|
Rename DP_RANK to SGLANG_DP_RANK (#2218)
|
2024-11-27 09:36:36 -08:00 |
|
|
Lianmin ZhengandYineng Zhang
|
fed4c6946a
|
Release v0.3.6.post2 (#2214)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2024-11-27 03:35:30 -08:00 |
|
|
Lianmin Zheng
|
fb6e04a0c2
|
Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2222)
|
2024-11-27 02:52:46 -08:00 |
|
|
Lianmin Zheng
|
6997e28f6e
|
Revert "Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default" (#2221)
|
2024-11-27 02:02:01 -08:00 |
|
|
Lianmin Zheng
|
a0e58740a8
|
Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2217)
|
2024-11-27 01:13:41 -08:00 |
|
|
Ying Sheng
|
37c8a5761f
|
[feat] Support session control for vision language models (#2210)
|
2024-11-27 00:03:29 -08:00 |
|
|
Lianmin Zheng
|
c754652fcd
|
Fix flasky tests (#2212)
|
2024-11-26 23:06:20 -08:00 |
|
|
Byron Hsu
|
0b46b951ae
|
Fix rust warning (#2208)
|
2024-11-26 15:00:41 -08:00 |
|
|
Byron Hsu
|
2763c0a73a
|
Bump router to 0.0.9 with better logging (#2207)
|
2024-11-26 13:30:28 -08:00 |
|
|
Yineng Zhang
|
de3b67b77d
|
docs: update adoption (#2204)
|
2024-11-26 12:57:16 -08:00 |
|
|
Yudi Xue
|
19f33b3237
|
add sglang version to get_server_info (#2206)
|
2024-11-26 12:10:23 -08:00 |
|
|
Yineng Zhang
|
30ce5b599e
|
minor: update check_env (#2201)
|
2024-11-26 18:22:55 +08:00 |
|
|
Yineng Zhang
|
bc1f6fda0d
|
fix: add cuda-python for xgrammar (#2199)
|
2024-11-26 17:24:18 +08:00 |
|
|
Wang Ran (汪然)
|
867e092f82
|
using is not not != to test None (#2196)
|
2024-11-26 01:00:38 -08:00 |
|
|
Andrew Lyu
|
88c7763f53
|
Remove unresolved reference 'self' (#2198)
|
2024-11-26 00:59:58 -08:00 |
|
|
Wang Ran (汪然)
|
e4118b15b3
|
remove unused imports (#2195)
|
2024-11-26 00:59:36 -08:00 |
|
|
Lianmin Zheng
|
ba4ee37fa4
|
Update sampler.py to skip the success check (#2197)
|
2024-11-26 00:58:57 -08:00 |
|
|
Lianmin Zheng
|
ac5a0f0488
|
Release v0.3.6.post1 (#2189)
|
2024-11-25 17:31:37 -08:00 |
|
|
Lianmin Zheng
|
ea34350d88
|
Rename double sparsity config file (#2188)
|
2024-11-25 17:12:08 -08:00 |
|
|
Lianmin Zheng
|
1605ae121e
|
[CI] Minor fix for CI (#2187)
|
2024-11-25 16:38:43 -08:00 |
|
|
Rin Intachuen
|
1aea19f64b
|
Input_embeds support (#2052)
|
2024-11-25 16:35:04 -08:00 |
|
|
Byron Hsu
|
1f76fc6e3f
|
[router] Rust e2e test (#2184)
|
2024-11-25 16:02:03 -08:00 |
|
|
Yixin DongandBen Gitter
|
7f076c2ce6
|
Update XGrammar to the latest API (#2176)
Co-authored-by: Ben Gitter <gitterbd@gmail.com>
|
2024-11-25 15:58:30 -08:00 |
|
|
Lianmin Zheng
|
3c5538f781
|
Update CI threshold (#2186)
|
2024-11-25 15:24:17 -08:00 |
|
|
HAI
|
10189d08dd
|
[Performance]: Process affinity to CPU cores with multiple sockets support (#2171)
|
2024-11-25 14:57:32 -08:00 |
|