3 Commits

Author	SHA1	Message	Date
Lianmin Zheng	de167cf5fa	Fix request abortion (#6184)	2025-05-10 21:54:46 -07:00
Lianmin Zheng	849c83a0c0	[CI] test chunked prefill more (#5798)	2025-04-28 10:57:17 -07:00
Lianmin Zheng	ac2387279e	Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) ... Co-authored-by: SangBin Cho <rkooo567@gmail.com> Co-authored-by: dhou-xai <dhou@x.ai> Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>	2025-03-03 00:12:04 -08:00