121 Commits

Include renames

Author	SHA1	Message	Date
Lianmin Zheng	41598e0d8e	Add longer accuracy test on CI (#1049)	2024-08-12 09:21:38 +00:00
Liangsheng Yin	7de6034534	Fix the prefix indices (#1037)	2024-08-11 17:57:02 -07:00
Lianmin Zheng	9dae407812	Improve type annotation (#1029)	2024-08-11 02:44:59 -07:00
Lianmin Zheng	a97df79124	Clean up readme and arguments of chunked prefill (#1022)	2024-08-11 01:18:52 -07:00
Liangsheng Yin	43fbb6d919	Fix input_ids && rename to fill_ids (#1021)	2024-08-10 16:24:12 -07:00
Lianmin Zheng	54fb1c80c0	Clean up unit tests (#1020)	2024-08-10 15:09:03 -07:00
Liangsheng Yin	62757db6f0	Reduce the overhead when cache is disabled (#1010)	2024-08-09 16:36:57 -07:00
Liangsheng Yin	73fa2d49d5	Some warnings to crash when CI (#1009)	2024-08-09 15:16:23 -07:00
gryffindor-rrandlzhang	9cf0a5bada	Add skip_tokenizer_init args. (#959) ... Co-authored-by: lzhang <zhanglei@modelbest.cn>	2024-08-09 12:14:13 -07:00
Ying Sheng	b16e856f11	Add openai embedding API (#997)	2024-08-09 11:19:18 -07:00
Ying Sheng	e040a2450b	Add e5-mistral embedding model - step 3/3 (#988)	2024-08-08 16:31:19 -07:00
Liangsheng Yin	2b8257f325	Adjust max prefix len (#980)	2024-08-08 00:41:26 +00:00
Liangsheng Yin	7623091d97	RadixCache method adjust (#977)	2024-08-07 15:52:24 -07:00
Liangsheng Yin	f724f1f1e9	PrefillAdder abstraction (#968)	2024-08-07 13:47:28 -07:00
Liangsheng Yin	a01ddd9605	misc: fix the req_to_token member change (#967)	2024-08-07 01:52:10 -07:00
Liangsheng Yin	7fa54a1ab3	Make req_pool_indices on CPU (#960)	2024-08-07 01:41:25 -07:00
Liangsheng Yin	87e8c090e9	Organize code (rename, movement) (#953)	2024-08-06 20:50:32 -07:00
Liangsheng Yin	ad56e68495	Fix stuck in get_new_prefill_batch (#948)	2024-08-06 01:05:58 -07:00
Liangsheng Yin	c020f9ceda	Support chunked prefill when radix cache is disabled (#811)	2024-08-01 00:29:01 -07:00
Liangsheng Yin	a6c7ebbbcb	Add req slots leaking check (#842)	2024-07-30 18:29:01 -07:00
Liangsheng Yin	cdcbde5fc3	Code structure refactor (#807)	2024-07-29 23:04:48 -07:00