 Liangsheng Yin
|
7523541962
|
model_rpc style improvement (#293)
|
2024-03-24 15:41:24 +08:00 |
|
|
Qubitium
|
ce216c80dc
|
Cleanup codebase: removed unnecessary code/logic (#298)
|
2024-03-23 10:15:16 -07:00 |
|
|
Liangsheng Yin
|
1b35547927
|
Organize server_args (#277)
|
2024-03-11 20:06:52 +08:00 |
|
|
Liangsheng Yin
|
89885b31ef
|
Gemma Support (#256)
|
2024-03-11 12:14:27 +08:00 |
|
|
Cody Yu
|
4cb9aaedf3
|
Fix logprobs with logprob_start_len (#193)
|
2024-02-22 10:33:03 -08:00 |
|
|
psych0v0yager
|
9de9a46815
|
Added the ability to Modify the Context Length (#210)
|
2024-02-20 16:22:56 -08:00 |
|
|
Cody Yu
|
63ba630bbb
|
Refactor decoding logprob and add completion_tokens_wo_jump_forward (#189)
|
2024-02-15 10:54:20 -08:00 |
|
|
Lianmin Zheng
|
6493256b7d
|
improve print
|
2024-02-12 12:43:48 +00:00 |
|
|
Lianmin Zheng
|
c51020cf0c
|
Fix the chat template for llava-v1.6-34b & format code (#177)
|
2024-02-11 05:50:13 -08:00 |
|
|
Cody Yu
|
4d303c4fa3
|
Fix token usage with jump forward (#174)
|
2024-02-09 20:06:15 -08:00 |
|
|
Liangsheng YinandJa1Zhou
|
b1a3a454ee
|
add --disable-disk-cache (#160)
Co-authored-by: Ja1Zhou <50169346+Ja1Zhou@users.noreply.github.com>
|
2024-02-08 00:50:12 +08:00 |
|
|
Lianmin Zheng
|
23f05005fd
|
Format code & move functions (#155)
|
2024-02-06 13:27:46 -08:00 |
|
|
Cody Yu
|
a7334aeea1
|
Support decode token logprobs (#130)
|
2024-02-06 12:24:55 -08:00 |
|
|
Liangsheng Yin
|
26f0bedc8f
|
jump-forward rename (#144)
|
2024-02-05 16:50:37 +08:00 |
|
|
Liangsheng Yin
|
bb3a3b6675
|
Support Faster JSON decoding for llava (#137)
When sending fast-forwarded reqs to model_rpc, re-calculate `pad_input_ids`
|
2024-02-03 23:32:05 +08:00 |
|
|
Ying Sheng
|
e095b16236
|
Add max_prefill_num_token into server arguments (#133)
|
2024-02-03 02:35:54 -08:00 |
|
|
Lianmin Zheng
|
c7af9f7393
|
Fix a bug in llava-hd
|
2024-01-31 18:52:15 +00:00 |
|
|
Lianmin Zheng
|
ad82bac6f5
|
Fix model loading & format code (#125)
|
2024-01-30 23:49:52 -08:00 |
|
|
Cody Yu
|
71b54eea7d
|
Add cache metrics (#119)
|
2024-01-30 22:13:14 -08:00 |
|
|
Lianmin Zheng
|
6f560c761b
|
Improve the control of streaming and improve the first token latency in streaming (#117)
|
2024-01-29 17:05:42 -08:00 |
|
|
Liangsheng Yin
|
81561f8e2d
|
Flush Cache API (#103)
|
2024-01-25 21:32:59 -08:00 |
|
|
parasol-aserandYing Sheng
|
23950056f0
|
support speculative execution for openai API (#48)
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
|
2024-01-25 01:57:06 -08:00 |
|
|
Liangsheng Yin
|
01ee0fbc05
|
fast regex decode
Auto-detect constant str path in regex FSM, then extend instead.
|
2024-01-25 01:16:25 +08:00 |
|
|
Lianmin Zheng
|
fa7a696d04
|
Fix max_new_tokens for limited memory
|
2024-01-24 10:44:32 +00:00 |
|
|
shiyi.c_98andHaotian Liu
|
c6576e820c
|
Llava-hd Support (#92)
Co-authored-by: Haotian Liu <liuhaotian.cn@gmail.com>
|
2024-01-24 01:51:21 -08:00 |
|
|
Lianmin Zheng
|
9a16fea012
|
Return logprob for choices (#87)
|
2024-01-23 05:07:30 -08:00 |
|
|
Liangsheng Yin
|
ca13f3b8c5
|
Disk FSM cache and adjust code. (#63)
|
2024-01-20 21:26:11 -08:00 |
|
|
Liangsheng Yin
|
40ab1f0129
|
Fix the possible bug of decode out of memory (#36)
|
2024-01-19 11:01:15 -08:00 |
|
|
Lianmin Zheng
|
22ec7bc2a1
|
Expose more arguments to control the scheduling policy (#32)
|
2024-01-17 18:37:02 -08:00 |
|
|
Lianmin Zheng
|
bf51ddc6e5
|
Improve docs & Rename Gemini -> VertexAI (#19)
|
2024-01-17 02:54:41 -08:00 |
|
|
Lianmin Zheng
|
c4707f1bb5
|
Improve docs (#17)
|
2024-01-16 19:53:55 -08:00 |
|
|
Ying ShengandLianmin Zheng
|
ffe4aaee1d
|
Fix for T4 GPUs (#16)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2024-01-16 15:49:03 -08:00 |
|
|
Liangsheng Yin
|
08ab2a1655
|
Json Decode && Mutl-Turns (#4)
|
2024-01-15 00:49:29 -08:00 |
|
|
|
22085081bb
|
release initial code
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: parasol-aser <3848358+parasol-aser@users.noreply.github.com>
Co-authored-by: LiviaSun <33578456+ChuyueSun@users.noreply.github.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
|
2024-01-08 04:37:50 +00:00 |
|