 Liangsheng Yin
|
d5de20a3ee
|
Fix sync() when fork(1) (#412)
|
2024-05-08 15:15:18 +08:00 |
|
|
Liangsheng Yin
|
14522e6a26
|
Organize Benchmark (#381)
|
2024-05-05 16:14:17 +08:00 |
|
|
Liangsheng Yin
|
95c4e0dfac
|
Format Benchmark Code (#399)
|
2024-04-28 21:06:22 +08:00 |
|
|
Liangsheng Yin
|
19818b9c2f
|
Minor: style improvement of radix_cache and memory_pool (#395)
|
2024-04-26 01:01:36 +08:00 |
|
|
Liangsheng Yin
|
9216b10678
|
Improve performance when running with full parallel (#394)
|
2024-04-25 17:29:07 +08:00 |
|
|
Liangsheng Yin
|
da19434c2f
|
Benchmark Updates (#382)
|
2024-04-24 02:23:01 +08:00 |
|
|
Liangsheng Yin
|
150d7020ed
|
Revert removing the unused imports (#385)
|
2024-04-23 22:36:33 +08:00 |
|
|
Liangsheng Yin
|
9acc6e3504
|
add .isort.cfg (#378)
|
2024-04-22 22:38:09 +08:00 |
|
|
Liangsheng Yin
|
1bf1cf1953
|
Reduce overhead when fork(1) (#375)
|
2024-04-21 17:25:14 +08:00 |
|
|
Liangsheng Yin
|
62b3812b69
|
Time cost utils (#355)
|
2024-04-09 23:27:31 +08:00 |
|
|
Liangsheng Yin
|
ed27a6b992
|
Revert "Eliminate 2 gpu ops during sampling when logit_bias is zero" (#345)
|
2024-04-03 12:45:01 +08:00 |
|
|
Liangsheng YinandQubitium
|
463c6632a8
|
Eliminate 2 gpu ops during sampling when logit_bias is zero (#343)
Co-authored-by: Qubitium <417764+Qubitium@users.noreply.github.com>
|
2024-04-02 19:14:55 +08:00 |
|
|
Liangsheng Yin
|
2af565b3bb
|
[model] DBRX-instruct support (#337)
|
2024-03-28 10:05:19 -07:00 |
|
|
Liangsheng Yin
|
3842eba5fa
|
Logprobs Refractor (#331)
|
2024-03-28 14:34:49 +08:00 |
|
|
Liangsheng Yin
|
24e59f5350
|
model_runner simplify (#329)
|
2024-03-24 19:48:37 +08:00 |
|
|
Liangsheng Yin
|
7523541962
|
model_rpc style improvement (#293)
|
2024-03-24 15:41:24 +08:00 |
|
|
Liangsheng Yin
|
1b35547927
|
Organize server_args (#277)
|
2024-03-11 20:06:52 +08:00 |
|
|
Liangsheng Yin
|
89885b31ef
|
Gemma Support (#256)
|
2024-03-11 12:14:27 +08:00 |
|
|
Liangsheng Yin
|
a7ace9c88d
|
Fix qwen config (#261)
|
2024-03-10 18:54:18 -07:00 |
|
|
Liangsheng Yin
|
dfb13ac455
|
Fix addr reuse in check_port (#253)
|
2024-03-03 17:09:16 +08:00 |
|
|
Liangsheng Yin
|
ec90b9c054
|
Upload agent_calls.jsonl download link (#226)
|
2024-02-24 19:03:46 +08:00 |
|
|
Liangsheng YinandEnrique Shockwave
|
c4e9ebe3a4
|
Fix stop str merging (#225)
Co-authored-by: Enrique Shockwave <33002121+qeternity@users.noreply.github.com>
|
2024-02-24 16:05:21 +08:00 |
|
|
Liangsheng Yinandcomaniac
|
91e036334f
|
Adjust outlines version. (#200)
Co-authored-by: comaniac <hao.yu.cody@gmail.com>
|
2024-02-17 13:40:39 +08:00 |
|
|
Liangsheng Yin
|
37b42297f8
|
import outlines (#168)
|
2024-02-09 10:13:02 +08:00 |
|
|
Liangsheng YinandJa1Zhou
|
b1a3a454ee
|
add --disable-disk-cache (#160)
Co-authored-by: Ja1Zhou <50169346+Ja1Zhou@users.noreply.github.com>
|
2024-02-08 00:50:12 +08:00 |
|
|
Liangsheng Yin
|
26f0bedc8f
|
jump-forward rename (#144)
|
2024-02-05 16:50:37 +08:00 |
|
|
Liangsheng Yin
|
bb3a3b6675
|
Support Faster JSON decoding for llava (#137)
When sending fast-forwarded reqs to model_rpc, re-calculate `pad_input_ids`
|
2024-02-03 23:32:05 +08:00 |
|
|
Liangsheng Yin
|
cd8c3ccd95
|
Fix is_multimodal_model judge (#132)
|
2024-02-03 11:48:01 +08:00 |
|
|
hnyls2002
|
9c121f2a45
|
minor fix: result dump format
|
2024-02-02 09:58:24 +00:00 |
|
|
Liangsheng Yin
|
79cb018e4b
|
Add city doc benchmark mode (#129)
|
2024-02-01 13:38:47 +08:00 |
|
|
Liangsheng Yin
|
81561f8e2d
|
Flush Cache API (#103)
|
2024-01-25 21:32:59 -08:00 |
|
|
Liangsheng Yin
|
01ee0fbc05
|
fast regex decode
Auto-detect constant str path in regex FSM, then extend instead.
|
2024-01-25 01:16:25 +08:00 |
|
|
Liangsheng Yin
|
ca13f3b8c5
|
Disk FSM cache and adjust code. (#63)
|
2024-01-20 21:26:11 -08:00 |
|
|
Liangsheng Yin
|
40ab1f0129
|
Fix the possible bug of decode out of memory (#36)
|
2024-01-19 11:01:15 -08:00 |
|
|
Liangsheng Yin
|
01ca82d765
|
fix radix cache match (#7)
|
2024-01-15 09:42:46 -08:00 |
|
|
Liangsheng Yin
|
08ab2a1655
|
Json Decode && Mutl-Turns (#4)
|
2024-01-15 00:49:29 -08:00 |
|
|
hnyls2002
|
f652494df1
|
fix typo
|
2024-01-10 04:21:17 +00:00 |
|
|
Liangsheng Yin
|
331848de9d
|
Add SRT json decode example (#2)
|
2024-01-09 12:35:44 -08:00 |
|
|
Liangsheng Yin
|
ead5b39f82
|
Add flashinfer && Oultines (#1)
|
2024-01-08 08:26:18 -08:00 |
|