 Cody Yu
|
b803b395b7
|
Disable graceful shutdown of tokenizer manager when not in the main thread (#2872)
|
2025-01-15 03:29:33 -08:00 |
|
|
Cody Yu
|
3c2c5869ad
|
Support outlines > 0.0.31 (#219)
|
2024-02-24 15:06:17 +08:00 |
|
|
Cody Yu
|
4cb9aaedf3
|
Fix logprobs with logprob_start_len (#193)
|
2024-02-22 10:33:03 -08:00 |
|
|
Cody Yu
|
2a74748b2f
|
Pin outlines version (#196)
|
2024-02-16 13:01:40 -08:00 |
|
|
Cody Yu
|
63ba630bbb
|
Refactor decoding logprob and add completion_tokens_wo_jump_forward (#189)
|
2024-02-15 10:54:20 -08:00 |
|
|
Cody Yu
|
50afed4eaa
|
Support extra field regex in OpenAI API (#172)
|
2024-02-10 17:21:33 -08:00 |
|
|
Cody Yu
|
4d303c4fa3
|
Fix token usage with jump forward (#174)
|
2024-02-09 20:06:15 -08:00 |
|
|
Cody Yu
|
cba5027332
|
Fix BaseCache metric (#170)
|
2024-02-08 17:23:09 -08:00 |
|
|
Cody Yu
|
26c3494152
|
[Submodule] Change FlashInfer to import (#156)
|
2024-02-06 19:28:29 -08:00 |
|
|
Cody Yu
|
a7334aeea1
|
Support decode token logprobs (#130)
|
2024-02-06 12:24:55 -08:00 |
|
|
Cody Yu
|
ccbe1e67d8
|
Temporary fix OpenAI API for Pydantic v1/v2 (#153)
|
2024-02-06 11:34:15 -08:00 |
|
|
Cody Yu
|
322421fae3
|
Add warmup to SRT server (#146)
|
2024-02-05 14:21:16 -08:00 |
|
|
Cody Yu
|
71b54eea7d
|
Add cache metrics (#119)
|
2024-01-30 22:13:14 -08:00 |
|
|
Cody Yuandjohndun
|
cd6872334e
|
Fix Mistral model loading (#108)
Co-authored-by: johndun <dunavent.jm@gmail.com>
|
2024-01-26 09:38:43 -08:00 |
|
|
Cody Yu
|
3a581e9949
|
Dynamic model class loading (#101)
|
2024-01-25 15:29:07 -08:00 |
|
|
Cody Yu
|
23471f9aa3
|
Support v1/chat/completions (#50)
|
2024-01-18 23:43:09 -08:00 |
|
|
Cody Yu
|
61d4c93962
|
Support stream=True in v1/completions (#49)
|
2024-01-18 17:00:56 -08:00 |
|
|
Cody Yu
|
dafafe5b11
|
Use HTTP link in 3rdparty module (#42)
|
2024-01-18 11:18:22 -08:00 |
|