Jerry Zhang 30b404ce72 Add torchao quant for mixtral and qwen_moe (#1418)

2024-09-14 06:46:55 +00:00

..

configs

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

constrained

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

layers

Add torchao quant for mixtral and qwen_moe (#1418)

2024-09-14 06:46:55 +00:00

lora

[Feature] Initial support for multi-LoRA serving (#1307)

2024-09-12 16:46:14 -07:00

managers

Optimize conflicts between CUDA graph and vocab mask tensors (#1392)

2024-09-13 20:27:53 -07:00

mem_cache

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

model_executor

Optimize conflicts between CUDA graph and vocab mask tensors (#1392)

2024-09-13 20:27:53 -07:00

models

Add torchao quant for mixtral and qwen_moe (#1418)

2024-09-14 06:46:55 +00:00

openai_api

Make stop reason a dict instead of str (#1407)

2024-09-12 20:47:31 -07:00

sampling

Optimize conflicts between CUDA graph and vocab mask tensors (#1392)

2024-09-13 20:27:53 -07:00

conversation.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

hf_transformers_utils.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

[Feature] Initial support for multi-LoRA serving (#1307)

2024-09-12 16:46:14 -07:00

server.py

[Feature] Initial support for multi-LoRA serving (#1307)

2024-09-12 16:46:14 -07:00

utils.py

[Feature] Initial support for multi-LoRA serving (#1307)

2024-09-12 16:46:14 -07:00