havetc ecb8bad276 Returning a per request metric for number of cached_tokens read (#1599)

2024-10-16 11:49:22 -07:00

..

configs

minor: fix config (#1524)

2024-09-27 01:49:16 -07:00

constrained

[FIX] Catch syntax error of Regex Guide to avoid crash (#1521)

2024-09-28 12:42:06 -07:00

layers

Support double sparsity (#1459)

2024-10-14 02:00:41 -07:00

lora

Use is_flashinfer_available to replace is_hip for flashinfer check (#1596)

2024-10-06 22:54:05 -07:00

managers

Returning a per request metric for number of cached_tokens read (#1599)

2024-10-16 11:49:22 -07:00

mem_cache

Add a new event loop (#1677)

2024-10-16 01:33:20 -07:00

model_executor

Support double sparsity (#1459)

2024-10-14 02:00:41 -07:00

models

Add OLMo model (#1676)

2024-10-16 00:11:18 -07:00

openai_api

Returning a per request metric for number of cached_tokens read (#1599)

2024-10-16 11:49:22 -07:00

sampling

[Minor] Add some utility functions (#1671)

2024-10-14 20:08:03 -07:00

conversation.py

Add image_token in conversation.py (#1632)

2024-10-11 05:07:51 -07:00

hf_transformers_utils.py

[Fix] Fix clean_up_tokenization_spaces in tokenizer (#1510)

2024-09-24 21:37:33 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

Returning a per request metric for number of cached_tokens read (#1599)

2024-10-16 11:49:22 -07:00

server.py

Launch a thread to overlap CPU and GPU (#1687)

2024-10-16 11:20:17 -07:00

utils.py

Add a new event loop (#1677)

2024-10-16 01:33:20 -07:00