Ke Bao 2c615d120f [Feature] Support fp8 e5m2 kv cache with flashinfer (#1204) ...

Co-authored-by: Yineng Zhang <me@zhyncs.com>

2024-08-25 17:38:11 -07:00

..

constrained

Fix jump forward final state circular path bug. (#1084)

2024-08-13 21:14:05 -07:00

layers

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

managers

[Minor] Improve the function organization in TokenizerManager & improve loggers (#1208)

2024-08-25 14:46:34 -07:00

mem_cache

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

model_executor

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

models

Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186)

2024-08-25 10:29:12 -07:00

openai_api

[Minor] Improve the function organization in TokenizerManager & improve loggers (#1208)

2024-08-25 14:46:34 -07:00

sampling

Improve code style of sampler (#1168)

2024-08-21 16:48:24 -07:00

conversation.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

hf_transformers_utils.py

[Minor] Improve the function organization in TokenizerManager & improve loggers (#1208)

2024-08-25 14:46:34 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

model_config.py

Support MLA for DeepSeek-V2 with Triton - step 1 (#905)

2024-08-05 03:40:33 +10:00

server_args.py

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

server.py

[Minor] Improve the function organization in TokenizerManager & improve loggers (#1208)

2024-08-25 14:46:34 -07:00

utils.py

[Minor] Improve the function organization in TokenizerManager & improve loggers (#1208)

2024-08-25 14:46:34 -07:00