 Ata Fatahi
|
2ac36b9a7b
|
Make request payload size configurable (#2444)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
|
2024-12-11 16:55:21 -08:00 |
|
|
Byron Hsu
|
6387098f5f
|
[router] add health checking in router init (#2393)
|
2024-12-08 17:17:37 -08:00 |
|
|
Byron Hsu
|
2a717c5078
|
[Router] fix interrupt from terminal (#2413)
|
2024-12-08 16:58:41 -08:00 |
|
|
Byron Hsu
|
a1e697b25b
|
[router] Improve cleanup logic (#2411)
|
2024-12-08 15:24:02 -08:00 |
|
|
Lianmin Zheng
|
2a02185c5f
|
Rename DP_RANK to SGLANG_DP_RANK (#2218)
|
2024-11-27 09:36:36 -08:00 |
|
|
Byron Hsu
|
4d62bca542
|
[router] Replace print with logger (#2183)
|
2024-11-25 13:36:02 -08:00 |
|
|
Byron Hsu
|
4b0a1c9365
|
Replace prob based with threshold based load balancing (#2170)
|
2024-11-24 23:17:11 -08:00 |
|
|
Byron Hsu
|
32293a299c
|
Improve sglang router (#2148)
|
2024-11-23 17:34:24 -08:00 |
|
|
Byron Hsu
|
cbedd1db1d
|
[router] cache-aware load-balancing router v1 (#2114)
|
2024-11-23 08:34:48 -08:00 |
|
|
Byron Hsu
|
86c37d010a
|
fix sglang_router not found (#2005)
|
2024-11-11 15:20:14 -08:00 |
|